I am known to go to great lengths and even greater distances to meet great bootmakers and shoe stores of some repute. Most people make vacation itineraries around the idea of seeing ruins or really old buildings. I prefer shoes and fantastic food (in that order) to be my guiding light.
It must be because as a child, I grew up with only two pair of shoes — one for school and another for play. Eating out became an option, only later in my teen years. Like, I often say, what you don’t have as a child defines you more than what you actually do have. So perhaps that is why when I go to a foreign city, I finish my work and find a few great restaurants. And then walk off the good food by walking to some little known shoe store (for rest of the world) or a cobbler of some repute.
A friend, once finding out that I was going to be visiting Tuscany recommended a shoemaker, for he lived a few miles from where I was going to be making a temporary residence. He said it was perhaps about 25 miles to the ancient city of Siena, which is well known for its annual festival, Palio di Siena.
Having gotten used to the American roads, I roughly equate 25 miles to about 30 minutes — thanks to fairly straight and wide freeways, especially in the midwest and western parts of the country. The speed limit too isn’t too bad and folks traverse great distances at a pretty decent clip. So when I think of 25 miles, I often think about the distance from San Francisco to Redwood City, which on a decent traffic day is about a 30 minute drive down US Highway 101.
With that in mind, I was looking forward to this quick drive to Siena. When, I looked at the Google maps at the start if the journey, Google Maps said that it was going to take about 57 minutes to travel 40 kilometers. In reality, the trip ended up taking about 80 minutes. The giant Google machine hadn’t really accounted for quite a few things machines aren’t supposed to understand — because they aren’t quantifiable.
For instance, as a US driver, you tend to get a lot more cautious on curvaceous and hilly roads, compared to Italians who throw around their little Fiats like go-karts on a racing track. Google didn’t account for one getting stuck behind a neon-colored lycra festooned bicycle group for ten minutes, before they could maneuver and give you room to pass. The machine also didn’t know that overnight some parts of the road has just crumbled away and well, driving became a lot more perilous.
- In this data driven world, how do we (and software and devices) deal with unexpected divergences between “the data” and the real world? Do we become so dependent on data-driven algorithms that some stale data can cause major havoc? How do we gracefully recover from such situations?
- How does the new world of data-driven intelligence not just provide quotidian services that make our lives more efficient, but create moments of serendipity that make our lives more joyful?
That picturesque drive (via SP 222, one of the most beautiful roads in Italy) through the Italian wine country was a great reminder of how much, rather, how little computers and data have context, both temporal and spatial. And it brought up two quintessential questions about the future of our data-shaped society.
The same distance means something else in a bus, in a straight line, or in my case, on SP 222, which like a silver snake slithers through the rising (and falling) contours of Tuscan hills. 40 kilometers in the US is very different from 40 kilometers in Tuscany or in Germany. Same 40 kilometers means differently to an American in Germany or an Italian in America. Similarly, 15 degrees centigrade is positively winter in New Delhi, but back in San Francisco that passes for a lovely day.
Both are a good reminder that context is everything and that context changes with every bit of additional data points. As data starts to shape our expectations and perhaps our experiences, one has to think about the role of randomness in providing some context to that data. How will data driven experiences — autonomous cars for example — account for the unknown, like the crumbled side of the road.
One of the reason I am fascinated by Google Maps and apps like Foursquare is not because they solve a very real problem, but instead, for me they are a living test bed of an Internet that is shape-shifting in real-time, is data rich and hyper-personalized to such an extreme that it can predict what comes next almost automagically. It is a network that uses connectivity to its extreme and offers the impossible.
Traffic and calculating its various permutations is actually a unique problem, though I suspect Google is most likely to be upto the challenge, thanks to its wide array of technology assets and its desire to play a role in the future of transportation, whether through autonomous cars, ride shares or some yet unknown experiment. A new class of network is emerging that invisibly strings together moving objects (cars, trucks, buses and other vehicles) giving them a relative context to each other.
Google owns Android and as such it has the ability to get a ping every so often from a moving Android device and can easily deduce speed. You can use this data to conclude the relative speeds and distances between various vehicles as well. Google, can also deduce if a US phone-number based device is being used in say Italy, and thus use that as a variable when calculating the speed and time to travel for an American driver. Data from Google-owned Waze can add even more nuance to driving conditions. Google recently bought a cube-sat company that will give its satellite eye view of the world on a near real-time basis. It is easy to foretell Google and its impact on the transportation business. Maps + Waze + Satellites + Cars = data that powers CarOS. In the future, carmakers pay a license fee for that CarOS.
I often think about the data rich world and wonder if we have to think less like scientists and more like data artists, who take every little detail into account. An extra stroke of green or a different shade of green can turn a grassy knoll into a lush meadow, or a woodland, a summer morning into rainy afternoon. In March 2013, I gave Uber a lot of stick for being soulless with their data. Earlier this summer, Uber’s Ren Lu in a blog post shared details of how Uber was trying to make predictions about their customers’ destinations. “Extensions of this project involve building more complex priors and likelihoods,” Lu writes. Those are the brushstrokes that change the perspective.
Where you get dropped off, where you subsequently order an Uber again perhaps in two hours and correlated with a local database, and the frequency of visits can give Uber a much better idea on predicting your next ride. Every additional piece of data for me is like that extra stroke of paint. The thinking here can’t just be more data — art is not about creating perfection, but instead it is about imperfect perfectness. Art is all about perception, reminding me of what Jonah Jones (who has spent past few years on Google Maps) once said: “There are no absolutes, its all perception.” Today’s world of data is about absolutes, often faulty and presumptuous.
Conversations today center about data analytics — where all people are looking to do is sort through the data they have, look for hidden trends, try and predict behaviors and then optimize. It is very cut-and-dry, antiseptic and without soul. I guess it is a pet peeve of mine — I keep bringing this again and again.
“Data, when its put up in front of you as a number, it gets stripped of the context of where the data came from, the biases inherent in it, and the assumptions of the models that created it,” said Sean Gourley who started Quid, a San Francisco-based data analytics company. “Data scientists are presented with a set of parameters to optimize over, yet they don’t take the time to step back and say, should I even be optimizing this at all. We need to step back from the scientific notations and start thinking of it as data intelligence.”
In the data rich world of tomorrow, all incoming data should come with a question it answers, adding more context and thus constantly enhancing the experience. The mapping apps are going to be at the forefront of this data experience movement. Sure in order to create these constantly updated, constantly refreshed and reshaped experiences, you need a lot of oomph. Widespread server farms, data processing engines and ability to get many different data feeds will be key to that future.
But this is not just about technical prowess, but is also about ability to weave together many different strings. In Florence, the city where SP 222 starts, I relearned the importance of polymaths. The marriage of art, science, divine and beauty were the gift of Florence, the cradle of renaissance. From that perspective, the data factories of today — Google, Amazon, and Facebook — have an opportunity to reshape our expectations and experience.
The number rule in my book is — Internet taketh & Internet giveth aka whatever is lost because of Internet is also replaced by a wonderful new gift from the Internet. If technology today has reduced actual humans to antiseptic clicks, users, engagement minutes and made everything “data” then the same technology has the ability to rekindle what is being lost in this data-infused world: subtle magic of discovery, so subtle that it is near invisible and yet brings a hint of a smile, unintentionally.
I am a sucker for maps — in fact I am a sucker for maps that tell me about the past. They are works of art — the colors and the shades, the thickness of the pen, the typography, make them much more interesting that essentially tools of navigation. They bring a sense of place and add the joy of journey. David Churbuck calls it “the metaphysics of mapping, of cartography, and of providing the stranger with a guide to the familiar.”
Google’s vision of the future should be progressing towards that metaphysical. Armed with sensors, data and live connection to the Internet, a map shouldn’t just be a map — it should a storyboard for journey yet not undertaken. It has to be essential for Google’s vision of the future — where they marry the social data of Waze and traffic reports, a future where cars will lack the personality of their predecessors and the journey will unfortunately be more (for the lack of a better word) plastic.
The sheer joy of the Internet is the speed with which choices multiply. The sheer agony of the Internet is the speed with which choices multiply. That is why we need this new kind of creativity. It has to figure out a way to nudge us to new things, and enable wonder and discovery. What would be cool — is not just the efficient data, but also small details that tell us about things we might find if we went off the beaten path, took a detour or two and let joy of discovery be the overriding factor. If this was a feature in Google’s CarOS, it would bring back some of the joy we associate with the experience of driving, and the serendipity. Of course, everything would be filtered through Google’s algorithms, but as long as Google doesn’t completely lose its humanity, we might just have to suck it up and deal with the world which is much bigger, larger and complex.
There is such a huge opportunity and also a massive risk of living in a data enriched world. Risk is because going to be left to the devices of those who have a singular point of view. I am often perturbed by the idea of someone using data to define our destinies. It is an opportunity, for it allows science and art to embrace curiosity and start to define a new kind of creativity.
What is that creativity, you might ask? To be candid, I have been thinking about it. I am first to admit, I don’t have any answers. I do know that it is not a filter bubble. I also am confident that it is not personalization through marriage of handful of numbers. I also know that it is subtle. It is also not more, but instead it is using more of everything and yet showing us less, showing when needed, without even knowing we need to see it. It will come not from science, but from imagination.
It is the kind of creativity you see in a seamless whole cut oxford shoe with no seams. The perfect leather, the clean simple look, the immaculate shape and quality. It is incredibly hard and the shoe itself is the detail. It tells the story, all the complexity simply hidden from the naked eye.
The quest for creativity is what got me to take a break from my vacation, and visit the wonderful shoemaker. We discussed philosophy, we talked about greed and ambition and more importantly we talked about shoes. Do shoes have a soul? Later, I fussed about the leather choices, the colors. In the end I ordered a pair. It is a shoe, like any other. But it will never be like any other. It is mine — it is unique, but more importantly, it has a story.
When I will look at those shoes, it will remind me of the drive, the quaint little shop, smells wafting out of trattoria next door and it will take me to a place that will make me smile. I would have reached my destination.That’s my kind of personalization.
Thank you Hiten Shah, David Churbuck, Anand Rajaraman and Derrick Harris for their feedback and suggestions.
Love it, beautiful weaving in and out of “data”, as a art, and data driven by context, data as number, things that I am grappling – what is the meaning, and what are the bias’, in a data driven world can we have “numbers” that leave us happy in the moment.
Thanks Ritu for your comment. I think we are going to eventually head to that point and see numbers start to make sense and get a sensibility.
Awesome skies and landscapes in autumn!
An excellent essay, very well written. I’d love to read a book or eBook of these, Om.
Thanks Rusty. Not sure I am there yet, but hopefully I have enough of them to turn it into a little booklet.
Wow what a beautiful post …actually a story ..would remember for years to come and i’m sure will refer back to this post for years to come … Once again thank you Om for an amazing post 🙂
Thanks @Mekalav.
lovely sky with lovely clouds. 🙂
Tuscany, lovely as ever. Om’s thoughtfulness, reflection – an appreciation of living fully.
Irrespective of the amount of data available, mankind is going down the path of a massive cultural shift where computers are making predictions and by extension decisions, which humans and other machines are accepting. As Peter Norvig wrote a few years back, it is the ‘rise of the machines’. This will liberate humans from the mundane and will unleash an era of unbridled creativity because that is one thing machines are pretty useless at. If humans view machine intelligence as but another tool to help us then our creativity will remain unbounded. Sure, there is a dark side to machine intelligence but there is to many human engineered tools.
Wrt the dryness (or lack of soul) of machine intelligence – you could equally ask, why do you want the predictions and decisions to have a soul? In fact, it can be argued that it is the last thing we want – let the intelligent machines be good at what they do and let humans be good at what they do (using the tool analogy). Also, once machine learning systems start becoming ubiquitous many startups will rise to the challenge of hiding the predictions behind fluid people-friendly interfaces.
As for the time to cross the hilly roads exceeding the static estimate, this and other similar problems will begin to be addressed once realtime scalable machine learning appears. Today, machine learning systems are static in that they are as good as the last data model update. It will be a long journey in an innovation at an exponential rate sort of way.
I love this, Om! It so in line with what I just wrote the other day about Data Anthropology and how this blend of art and science is happening. It’s here if you want to take a look: https://www.linkedin.com/pulse/article/20140910202003-4718724-in-the-era-of-brilliant-alchemy-data-anthropology
Maps are also history – on a country and global level, and on a personal one. What if we encouraged people to create maps of their lives, with stories and facts, photos and videos, impressions and reactions, that we could layer ‘person-map’ upon ‘person-map’ to create a richer understanding of each other and the world at large? A girl can dream…
That is such a wonderful concept and idea. It needs more imagination but will have more human context to it.
What is that creativity, you might ask? Creativity may emerge when we start to value the art of presence more than the science of productivity, especially when looking through this data driven lens. The current data science continuum is to describe-deduce-predict-optimize-prescribe, through which new number insights are communicated in the hope to make us more productive. However, when we move beyond this to having data-narratives or explorable-data-explanations shared with us and the meaning within them is extracted by us, then there is scope for the creativity within us to emerge.