Motion sensors such as accelerometers were initially embedded into smartphones to simply tell the system if the device was being in held in portrait or landscape view. I would hazard a guess that the engineers in those early days did not imagine that the acceleration signals extracted from the sensor would end up being used to characterize a wide assortment of other everyday activities; how many steps you walked, how many minutes you ran, sat in a car or even went cycling are all behavioural estimations that can be made with just that single accelerometer with the help of a bit of algorithmic wizardry. Nowadays, we take it for granted that there are many quantified-self analytics that come with every smartphone and new devices such as bracelets, smart watches and other wearables, which have been created to collect even more signals that log the happenings in your everyday life.
Indeed, behavioural sensing technologies that use motion, locational and physiological sensors are now a global industry. It is clear how heart rate and oxygen levels coupled with activity monitoring can be informative to those conscientious about their fitness, or a smart bracelet worn by a vulnerable person with walking frailty can alert caregivers when a fall happens. There is now a such wide variety of apps and devices that offer such solutions in this physical activity domain that I would not be able to list them all here. It is fairly easy to see why they are effective in their tasks: physical sensors are designed to sense physical things. But why stop there?
Physical and physiological phenomena are also the currency upon which we can infer someone’s internal experience. For example, non-verbal communication such as the meaning of a frown or a smile is learned by babies before they can communicate verbally. High sweat levels can occur because it is a hot day, but they could also occur with high intensity activities or emotions. We are always signalling in one way or a another how we are feeling.
So, we naturally ask the question could smart devices be used for the mental or emotional further to just the physical? Could we go further than just monitoring how far someone cycled, or if a frail person has fallen? Could we tell if music at a party is being well-received by the guests, or whether someone with a serious mental illness is relapsing? Yes, we can.
Systems that are designed to infer levels of enjoyment, emotional state and even changes in mental wellbeing have become increasingly more accurate. Moreover, in some applications the commercialization of such systems has also begun to emerge. The platform created by HealthRhythms makes use of a wide range of device sensors over long durations to ascertain changes in behaviour for the purpose of monitoring changes in mental health state. Mental health professionals can now add smart device sensing to their traditional therapeutics and pharmacological solutions.
Even in less clinical settings, device-based behavioural sensing is now being used. You can now upgrade your house party with the recently launched GroovSense app. Despite there being a range of other social jukebox-style apps currently available, GroovSense uniquely detects expressions of enjoyment within a crowd such as the number of people dancing. This combined with prior information on the guests’ music preferences and the capacity of a computer to search and sort through vast music libraries in seconds, GroovSense offers dynamic playlist optimization that reads and follows the mood of the room.
So, what’s next?
In my view the sensors we currently have are just the tip of the iceberg. The much-hailed emergence of the era of Internet of Things underpinned by 5G communications infrastructure will allow for everyday behaviours to be sensed and understood at a granularity and richness that is quite impossible today. Also, these advances with ever-more powerful device processing and sophisticated data science will mean much less need for private, sensitive data to be transmitted beyond your four walls. At that stage it may be more appropriate to speak less of a ‘quantified self’ and more about ‘quantified everybody’!
Of course, with such seismic advances in processing power, communications and artificial intelligence predicting what will happen in the long term maybe somewhat presumptuous. I place myself back in the shoes of those early engineers who placed a humble accelerometer to simply sense phone rotation, who couldn’t have known what it would eventually lead to. One thing is clear: systems such as HealthRhythms and GroovSense will be well-placed to lead the way in the application of passive behaviour sensing technology – and know how you are doing, not just what you are doing.
