What will change everything?

This question was posted on the www.edge.org

"What game-changing scientific ideas and developments do you expect to live to see?"

My Answer:

The evolution of dark data. Dark Data is the human data info sphere which is analogous to what physics currently labels as Dark Matter in the Universe. The currently unknown aspect of the universe which occupies the majority of it's volume. Like Dark Matter, Dark Data is expanding. In the age of the Internet the expansion of Dark Data is exponentially beyond our current ability to organize and interpret it very effectively AND efficiently.

I believe that in my lifetime we will create improved ways to bring light to the Dark Data. Methods will be employed to organize, visualize and interpret the newly lighted dark data. At first these methods will be laborious (like mapping the human genome was), but over time the methods will require less and less human intervention as at first human minds develop processes & systems to interpret the data....and then the machines enhance and develop additionally more effective and efficient ways to bring the dark data into the light.

The closest analogue to this right now is the BI/DW industry, currently requiring huge amounts of human and physical capital to convert dark data into information. The cost is dropping as more efficient ETL tools are created to Engineer, Transform and Load the data into a Light-data format for human perusal. Over time the process of ETL will become more automated as the collective intelligence regarding dimensional data schema and the underlying ETL process is catalogued for reuse.

Imagine the effect of truly granular information placed in the context of time and other levels of granularity in a format that is easily visualized in multiple ways at the discretion of the information consumer. Imagine that this information is also contextually aware, meaning if your walking down a street and feel like eating pizza that you merely have to say the words "I'm hungry for Pizza" and the technology available to you at that instant is able to correctly interpret that and place in your view any Pizza eatery within 10 minutes of your current location, along with the menu's, prices and estimated time of arrival....even allowing you to place the order and pay for the meal before you arrive. Upon arrival your custom slice of pizza is waiting for you timed to be ready at the time of the ETA your personal system communicated to the vendor.

Becoming a locationally-aware node within the larger dimensional schema of the web will involve the loss of some privacy with the payback being uber convenience.

Small business owners will be able to know every detail about there business at the appropriate contextual moment of need. More likely than not, the system(s) interacting with the available internal data will already have offered suggested actions requiring human intervention.

To me...this type of data ubiquity combined with high utility will occur in my lifetime and be truly transformational and since I like pizza quite a bit (I'm a pizza snob)...useful.

Blogging Your Life in Realtime

I found this link on Kevin Kelly's blog which is quite interesting with many social implications. The speaker is Ryan Grant the founder of an early stage start up called Metascopic Inc.

Ryan's goal is to make recording and retrieving the daily activities of your life very easy through the use of a small wearable device which would take up to 1,000 stills/day and provide the ability to record audio or video for at least some of the time slices in your day. Are you bad at remembering the names of people you meet? Imagine having a wearable device that would record all of that and a companion web service that you would upload your daily journal of activities to. Then image having a tool to index and retrieve that information efficiently.

That is what Metascopic is trying to develop. There are obvious privacy issues and social boundaries that need to be redefined for this practice to be accepted. Interesting idea which also shows the direction we're heading with ubiquitous records and the web.



QS_081023_02_Ryan_Grant from Paul Lundahl on Vimeo.

Tweepsearch & Evernote - Interesting Combination

This morning I read a post by Louis Gray http://tinyurl.com/auzu6q discussing a new Twitter search tool created by Dan Cortesi at /http://dcortesi.com/
Mr. Cortesi's tool provides a good way to search your Twitter follower list.

The search can be sorted by post date of your follower's lasted post or alphabetically by name.

I decided to experiment using Evernote with Tweepearch by clipping all my followers to my Evernote Desktop application. It will be interesting to see how I can use a fully indexed version of my Twitter follower listing. Tweepsearch pulls down the bio information which includes the location data.

I would really like to see Mr. Cortesi add an extract feature which will provide a way to pull the complete follower information (including the date that person started to follow) in a single flat file format like CSV or TXT for future analysis.

Thank you Dan Cortesi for providing this tool. Nice work.