Thursday, January 17, 2013

Inventing Hari Seldon's psychohistory - and other science marvels

The Defense Department wants new computer tools to analyze mounds of unstructured text, blogs and tweets as part of a coordinated push to help military analysts predict the future and make decisions faster. The search is part of the Office of Naval Research's "Data to Decisions" program, a series of three-to-10-year initiatives that will address the volume of information that threatens to overwhelm planners in the digital age. The ONR is calling for computer algorithms to predict events, fuse different forms of information and offer context on unfolding events. The office expects to spend $500,000 each year in funding.

Isaac Asimov_1951_FoundationThis is just one of many such endeavors currently underway, the most ambitious being a European Union plan devoting billions of Euros to developing predictive analytics that sound a lot like the "psychohistory" of Isaac Asimov's Foundation Universe and its genius prognosticator Hari Seldon.  (I know about such things: I entered Seldon's mind, in my novel set in Isaac's cosmos in my novel, FOUNDATION'S TRIUMPH that tied up many loose ends in the Foundation universe.)

Alas, while better tools for appraisal and projection are desperately needed (I portray them in EXISTENCE), I also consider all these expensive European and American efforts to be premature and likely doomed.  

Instead, what would be far better is to lay the foundations by doing one simple thing -- performing a broad-spectrum survey in order to find out who - in human civilization - happens to be right a lot.

SignalAndNoiseCan you believe that there has never been a systematic effort to do that one, simple thing?  Shine light on all the actuaries, horseracing touts, stock analysts, political pundits and so on who claim to have a handle on the future, with a clear and do-able aim - to appraise and score them, so that we can find out - at long last - who gets it right more often than others... perhaps anomalously often, far above chance? And in contrast, who is full of bull? Can you think of any appraisal... any at all... that could have more substantial positive effects on society and the world?  Or that would seem more terrifying to the "chattering castes"?

And so why do I push for it?  Guesses, anyone?

PredictionsRegistryI discuss elsewhere the many advantages. As well as complications, e.g. that the very best prognosticators might not want scrutiny applied to their methods. Still, we'd all have a chance for a better society if the best were scored and revealed and their methods scrutinized. It would also cost 1% of the vast (and futile) modeling programs under proposal. Ah, but perhaps that is why my suggestion is never taken seriously.  See: Accountability for Everyday Prophets: A Call for a Predictions Registry

== How to keep growth going... ==

Is growth slowing?  Every decade since the 1940s featured some cluster of new technologies -- from jets to satellites to pharma and the internet -- that created a burst of new wealth and productivity. (Which incidentally helped Americans to pay for the tsunami of purchases that then propelled world development.) I've pointed out that most U.S. problems are rooted in one sad fact: there was no such "big new thing" in the 1st decade of the 21st Century.

GrowthThat's partly the fault of troglodyte political leaders who ruled us then. But in a broader sense it provokes some to claim that the era of tech driven growth is over,  Paul Krugman replies in the NY Times that it's too soon to count technology out.  Still, even optimists should be concerned: while smart machines may make higher GDP possible, “they also reduce the demand for people — including smart people. So we could be looking at a society that grows ever richer, but in which all the gains in wealth accrue to whoever owns the robots.”

So, is the "maker movement" going to rescue American manufacturing independence... and civilization in general... as some  tech-utopians not predict? (And as I depict in my graphic novel TINKERERS.)  Have a look at a very thoughtful essay in Technology Review that considers some factors that the tech-transcendentalists - in their zeal to believe - may have missed.

Fortunately, the paucity of great economy-driving inventions dows not apply to pure science. A list of scientific and technical accomplishments made in 2012 suggests that our time of wonders is not behind us, but ahead.  Giving vision to the blind, discovering rogue planets or others made of diamond ... or orbiting the nearest star. Hacking the brain.  Photographing DNA. Invisibility cloaks and more...

== Saving the World... with self-driving cars? ==

nrdcClimate change increases the risk of many types of record-breaking extreme weather events that threaten communities across the country. In 2012, there were 3,527 monthly weather records broken for heat, rain, and snow in the US, according to information from the National Climatic Data Center (NCDC). That's even more than the 3,251 records smashed in 2011—and some of the newly-broken records had stood for 30 years or more.  More ammo for your idiot uncle to frantically dodge and deflect.

Okay, so what do cars have to do with solving this? Recall a couple of postings ago I described my own small role in the Clean Air Car Race of 1970, which might have hastened the removal of lead from gasoline. Now comes the endeavor of another Brin... Google's self-driving car, which may offer not only convenience, but a decline in the need for personally-owned automobiles, overall.  A problem for Detroit, but a boon to urban centers, if you can share a car with a dozen others and use your phone to summon it to come get you, whenever you need a ride.  Or join a coop or a flash-rental agency to get the same thing.  Or unleash your own car at night to roam around as a taxi, earning you bread?

== Science Potpourri ==

Kaggle attempts to use crowd-sourcing and contests/prizes to help small companies solve knotty software problems quickly. Companies pay to post their competitions to Kaggle. But those fees will be waived for the five startups Kaggle selects to take part in its new experiment in crowdsourced algorithms.

negative-temperature-atomsThe bizarre notion of "negative temperatures"  seems weird and quasi-mystical... though those claiming to have achieved it do not define temperature the way most of us do, as a property of atomic motion, but rather as a thermodynamic quality. From my understanding of what they are doing, it sounds rather like the inverted energy-state system you get inside a laser, plus some mumbo terminology... plus a trick of creating atoms that each have a "maximum energy state"... then causing the number of occupied high energy states to outnumber the low energy ones.  When all of the system's energy is crammed into those maximum energy states, the entropy of the system actually falls. That is, by strict, statistical definition.

See also this take on it. What the article doesn't explain is how the authors blocked  the system from simply adding higher states as happens in regular physics, all the way to ionized plasma. The researchers achieved this at nano-temperatures near absolute zero, which suggests taking advantage of some quantum-level barrier that can only be used in such extremely cold ranges.  Hence all the talk of "practical uses" seems, well, far fetched. In that light, "negative temperature" is a thermodynamics legalism. In fact, well, I feel frosty toward it.

Meanwhile... Australian Zircons have long been considered the oldest tracers of geology on Earth.  But this one apparently formed when the planet was just 150 million years old. In there are more surprises...tales of the new born earth.

== Bio Miscellany! ==

UKPostmanPBWe are definitely entering the era of "augmented soldiers" that I forecast in The Postman. Studies are underway, how the Pentagon and others might negotiate the minefield of risks and opportunities ahead.

"micromort" is a unit of statistical chance of increasing your likelihood of dying by one millionth (or decreasing it.)  An interesting approach to risk assessment.

A new device about the size of a business card could allow health care providers to test for insulin and other blood proteins, cholesterol, and even signs of viral or bacterial infection all at the same time — with one drop of blood.

On 23 occasions over the past several years, wild dolphins were observed giving gifts to humans at the Tangalooma Island Resort in Australia. The gifts included eels, tuna, squid, an octopus and an assortment of many other types of different fin fish.

And while we're at sea... Japanese researchers post first video of a giant squid in its natural habitat.

Release the kraken! A collection of tales and legends, poems and fiction, from Verne to Tennyson -- about this marvelous cephalopod.

The detailed changes in the structure of a virus as it infects an E. coli bacterium have been observed for the first time. 

Instructing scar tissue to change itself into healthy tissue. By using a cocktail of three specific genes researchers have used gene therapy to reprogram the scar tissue cells on a damaged heart into functional muscle cells, while the addition of a fourth gene stimulated the growth of blood vessels to enhance the effect.

The doctor's office of the future: a kiosk at your local grocery store?

 == And Technology on parade ==

Russia sets its sights on the moon: Four robot probes due for 2015, a manned mission by 2030. The first flight, slated for 2015, will see a 1.2 ton lunar lander called Luna-Glob (Moon Globe) deposited on the moon's surface to search for water, take soil samples, and beam back its findings to Earth. I wish Phobos-Grunt had succeeded... Phobos is valuable. Not so sure about the rest of this.

A Topological Recipe Book for New Materials. Researchers showed that they can create a recipe book to build new materials using the mathematics of topology (whose properties that do not change when an object is continuously deformed.)

I've always been a junkie for bridges.  I love every variety and find them the most esthetically wonderful things we humans make, blending art and nature's laws.  Now see how smart design and super materials are enabling ever longer cable-stay bridges.

Researchers used electricity on certain regions in the brain of a patient with chronic, severe facial pain to release an opiate-like substance that’s considered one of the body’s most powerful painkillers.

Construction is complete on behemoth airship; first flight planned. Worldwide Aeros, a company of about 100 employees, built the prototype under a contract of about $35 million from the Pentagon and NASA. The Aeroscraft is a zeppelin with a 230-foot rigid skeleton made of aluminum and carbon fiber. It's a new type of hybrid aircraft that combines airplane and airship technologies.

North Dakota is now brighter than Pennsylvania, as seen from space. Why? The fracking boom has so many oil rigs at work, flaring off excess natural gas under super-lax state rules, that ND glows like New York. Ah, yummy greenhouse. This needs adjustment. And soon.

UCSD has introduced Diego-san, a new humanoid robot who mimics the expressions of a one-year-old child. Diego-san’s hardware was developed by two leading robot manufacturers: the head by Hanson Robotics and the body by Japan’s Kokoro Co. The project is led by University of California, San Diego research scientist Javier Movellan, who directs the Institute for Neural Computation’s Machine Perception Laboratory. Well into the Uncanny Valley...

Read an update on the Google Glass Project, which promises to deliver heads-up, augmented reality (AR) to smart glasses you'll wear on the street... supposedly in 2014.  And to see where all this will lead?  Have a look at your future in Vinge's  RAINBOW'S END or (even more accurately) my EXISTENCE.

Challenging literature is good for the brain? Better than self-help books. Maybe even try some challenging Science Fiction titles....

Nanowire Arrays for Better Piezoelectric Energy Generators. Researchers developed a nanogenerator consisting of an array of vertically aligned nanowires that, when deformed by an impact or twist induces a piezoelectric production of electrons. The proof-of-concept work included producing enough energy to turn on an LED light..

And finally...  Ever had one of those chilling "coincidence" moments when Some set of flukes coincide?  "What're the odds of THAT happening?"  Are you familiar with The Odds Must Be Crazy website?  They collect these kinds of coincidences and attempt to de-mystify them with science and statistics.

David Brin
Twitter                Facebook


Doris said...

David listed "actuaries, horseracing touts, stock analysts, political pundits" and asked who was more accurate.

Of these, employers of actuaries and touts lose money if their oracle is wrong. They tend to dump bad oracles. Clients of stock analysts also lose money, but somehow the clients keep getting fooled. As for political pundits, accuracy is not what keeps the fish hooked, only a world view craved by the fish.

I'd go with the actuaries.

Dan Eastwood said...

Nate Silver =?= present-day Hari Seldon

The best predictors will not only make their methods public, but scrutiny will make them better still.

Anonymous said...

Is the problem that liars get to propagate so much bad information?

Come off it! We are supposed to believe that economists don't have a clue about planned obsolescence 43 years after the Moon landing? So how do you predict things when large percentages of the population are either ignorant or believe lies?

Blow away the lies and see if future history does not become something different.

Anonymous said...

micromort - sounds so much more scientific than the old "lightning strike" unit.

Janus Daniels said...

Krugman has a fair record and he (as well as other economists) specifically credited Asimov's psycho-history for leading to economics as a career choice.

Kevin Johansen said...

Hi David,

Specific to econ development - or as we call it, 'evidence based entrepreneurship' - we're building out a 'business development graph' that will also function as a best practices aggregation engine.

Ping me for details if you're interested, as I'd love to have your insight on what we're doing.

Kevin Johansen, CEO

Ian said...

If we had efficient large -scale energy storage couldn't the gas flaring actually be put to good use?

Paul451 said...

I find the giant squid way less creepy than the robot toddler.

Re: Robot's on the moon.
It has been a long time between drinks. And it would be nice if someone would finally put a lander/rover on the poles to look at the supposed ice. Someone might want to use it one day, it would be handy to know what its properties are. (Is it ice-coated dust, or mixed dust grains and ice crystals, or a layer of frost, or a nice thick layer of solid ice? Each would require a different collection and extraction method.) And given that the moon is within direct teleoperation range, and biocontamination is not an issue, it's mad that we don't buy a bunch of cheap disposable missions (in the style of COTS). Frankly anything but another Mars mission.

Re: Bridges.
I've been hoping the design of London's Millennial Footbridge would lead to new designs for major bridges that skipped the high towers and used clever lateral cabling under the decks instead. Maybe the wobble put designers off. (And I loved the Falkirk Wheel.)

TheMadLibrarian said...

It's too bad the petroleum industry can't take a lesson from the meatpacking industry: everything but the squeal. Probably it's not sufficiently cost effective to capture the gas from oil drilling and sell it, sadly.

hlkagr: remnants from a fracking process

Ian said...

Gas-flaring looks spectacularly but how much gas does it actually consume?

LarryHart said...

locumranch, just so you know, I've continued to respond to the gold-standard argument in the previous post, and am willing to continue doing so as long as you are.

LarryHart said...

Nate Silver =?= present-day Hari Seldon

The best predictors will not only make their methods public, but scrutiny will make them better still.

And on the other side, it was heartening to see Karl Rove get called out by (of all things) his own FOX News people. "Is that real math, or math that you do as Republicans to make yourselves feel better?" That was a long time in coming.

Jumper said...

Re predictions, I see two distinct strategies. In one, the predictions are held close to the vest. Insider trading; stock strategies in general, in particular genetic algorithms, won't work if everybody and her sister also know. And if we find out we can predict a crime do we publicize it and thus prevent it, or keep our knowledge secret to catch perps?

The point is there is a recursive element, often referred to as a self-avoiding prophecy. So I am unsure if "prediction" is exactly the right concept. In short, the use of analysis to predict only works in secret; if the wide world web watched publicly as the databases were scoured, no predictions would be reliable.

Jumper said...

Here's the core link to the topological colloids research.

I think this might relate to some of the elementary fiddling around I've done with cellular automata on asymmetric but regular grids. At least is a good point for people griping about funding "pure math."

Jumper said...

"colloidal particles of higher topological 'genus' or g value—from doughnut-shaped g = 1 through 'pair of glasses' g = 2 to g = 5—have been synthesized and studied. When introduced into a nematic liquid crystal, these colloidal particles align and generate three-dimensional director fields and defects in the liquid crystal fluid that can be manipulated with a variety of methods and their charge characteristics studied. "

David Brin said...

Oy, what a monster... among many such. Show me the liberal equivalents.

There may be some -- but as innumerable as locusts! In state legislatures across america?

shagggz said...

Worldwide Aeros mile high club = Worldwide Eros

Ian said...

Nate Slver reminds me less of Asimov's foundation series than of his short story "Franchise".

Hank Roberts said...

I'm guessing Google is trying to find what Avram Davidson called "The Sources of the Nile" now.

Track enough people asking related questions, assemble the pattern, figure out where those people are headed before any of the individuals involved do.

Then snap up their IP, hire the individuals, and claim to have created the idea that's the result of their questions.

Hank Roberts said...

> Gas-flaring ... how much


Anonymous said...

The O'Neal email showed poor judgement, though accusing him of desiring harm to come to Obama and his family seems nearly as overblown and childish as O'Neal's attitude toward Obama.

The most reasonable interpretation is that O'Neal saw sly humor in an apparent pro-Obama prayer request that actually expressed distaste for Obama, and lacked the judgement to simply giggle a bit and hit delete.

It's hardly to his credit, if he thought it just meant "I can't wait for Obama's term to be over" and didn't know or check the context of the following verses.

The Secret Service, professionals charged with investigating even semi-credible threats to the President, appear to have recognized his idiocy and ignored him.

So should we, EXCEPT for the people of Kansas, who should make O'Neal's embarrasing days in office short by recalling him.

To encourage that, we need a new bumper-sticker:
"Speaker O'Neal: Proverbs 18:6-7"

Anonymous said...

hm... I've recently had a short story published about predicting the weather, which is but a subset of predicting the future. It brought out the interesting poitn that different nations would prefer different futures and that there will consequently be skirmishes over small controllable events to make sure the a country gets what it wants... is this going to be the war-ground of the future?

Jonathan S. said...

The link to "The Odds Must Be Crazy" seems to be broken.

What are the chances, eh?

Tony Fisk said...

'Negative' temperatures are an outcome of the math used in statistical mechanics of paramagnetic materials and was being covered in undergraduate physics courses 35 years ago. They're considered < 0 due to lowered entropy, and hotter than infinity due to the inverted energy state occupancies

Here's a more humorous way to envision 'micromorts', Gunnerkrigg style (no-one's yet owned up to Paz for that stunt!)

George said...

link to odds must be crazy should be:
instead of

flahute said...

This is the second time I have read that there was no "next big thing" in the first decade of this century. Does the wide spread availability of handheld, multitouch computers like tablets and smartphones not rank as big enough?

I realize that it's not rocket science...

Miguel said...

Contrary Bin,

There is a real life recapitulation of Asimovian psychohistory underway now on Earth: by "Foundation Encyclopedia Dialectica [F.E.D.]".

They have derived a set of seven 'Psychohistorical-Dialectical Equations", using the "contra-Boolean Algebra" that their founder discovered, and axiomatized. --

However, although it is clear that they support and intend the use of "Big Data", their initial version of "the psychohistorical equations" are rooted in a different approach than mere statistics and probability theory.

Their initial equations are aimed at establishing a generalized, deterministic, necessary order-of-appearance, or order-of-progression, of "socio-ontology", an undergirding, causal "chain of socio-historical phenomena", around which "Big Data" can supply detailed, quantitative specifics.

For more about this, see --