Reality Mining at Cogito

Cogito Health is a young technology startup based in Charlestown, MA developing tools for real-time monitoring of customer engagement. Though their way of operationalizing engagement differs in marked ways from the game-centric way we usually approach the term here at Motivate.Play, I was curious to learn more about what they're working on, and how it might connect to the interests of our readers. Thanks to a personal connection (an alumnus of my lab at IU is on staff at Cogito), I was able to get an interview with the company's CEO and founder, Joshua Feast. Here's what he had to say.

Joshua Feast: When you say engagement, what do you think of? What does it mean to you?

Jared Lorince: That's actually the first question I had for you. It depends on context, right? When we talk about engagement on our blog and in the related research, we talk about players being engaged in the game, getting into the flow state, being totally engrossed in gameplay. That's how we talk about it, but I'm assuming you guys are operationalizing it a little differently?

JF: That's true. We often talk about engagement in the sense of the relationship between a customer and an organization. We tend to provide tools for organizations to manage their customers better, to work with their customers more successfully. It's similar in the sense of involvment, ...it may be involvement in a specific conversation, but it may also be involvemnet in healthcare self-management, and partaking in activities and things like that. That's one way we think about it. But I think it's somewhat similar to being engrossed, though perhaps in a longer timeframe. The only other way we think about engagmeent, which is slightly more unusual...is really trying to think about what someone is likely to do next. If someone is highly engaged, they're more likely to follow through with particular actions. So for example, if you're talking about medication adherence, if you have someone who is highly engaged, they're more likely to stay on their medications and things like that. It's a present involvmemnt in a future propensity to take part in an action.

JL: So is it a question of what signals are going to preidct that they're going to continue to take their medication, or to stay on the phone, or whatever the context? Or do you actually think about ways of enhancing the experience...again going back to questions of gamification, of how to make the experience more fun or enjoyable?

JF: We're very much focused on how we can improve the interaction between two human beings in a coversation. So that's very much what we try to do. We tend to work more on the organization side. E.g. providing real-time response (probably where the link to games is most obvious) to somebody on the phone to help them understand how they're being perceived when they speak, but also picking up on cues about the party they're talking with, about whether that person is on board with what they're saying, or distracted, or not interested. So you can maybe pull a conversation back if you're potentially losing somebody. So that's a huge part of what we try and do. And it becomes interesting when you compare to games, because a lot of the design problem in that is similar to some of the design problems in creating heads up displays.

JL: This gets a two particualr questions I had, the first being specifically what kind of signals you''re looking at. Obviously on a phone call you only have auditory signals, but what else do you use?. Related to that, what kind of feedback are you giving the agent? How does it work? What kind of interface is it? How real-time is real-time?

JF: Real-time is during the conversation as you're speaking. It will tell you, for example, if the conversation is unbalanced (one party speaking too much or too little), whether you are speaking fludily or not, whether you're speakign in a flat manner or dynamic manner. And also whether you're getting reactions from the other person, which relates to their mimicry behavior, you're mimicry behavior...how much they're paying attention, things like that. Typically we're focused on how people are speaking, and giving that information back to the agent so they can essentially have better understanding of social signals.

JL: So you're analyzing their vocal patters...has your approach been a black-box machine learning approach, or grounded in theory with expected correlates of engagment?

JF: Good point. Blunt force machine learning doesn't get you very far in this context; you have to come at it with theory. A lot of the signals we're looking for are based on evolutionary biology. Essentially we're tyring to get at substrate of what we call honest signalling, and that's basically a communication substrate that we evolved pre-language...Another major part of it is how fluid and how consistent someone can control their speaking because that indicates mastery of a topic or confidence. When someone is pausing and stuttureting, it means they're tired or distracted, or something of that nature. So those are the major patterns that are important. And they're pretty well-known in the evolutionary biology circles and, we generally try to pick them up and visualize them so people during a covnersation can be aware of that stuff even when they're not necesarrily paying attention. Often you're very much focusing on content and not paying attention to the social underpinnings of the conversation.

JL: So is everything these pre-language signals, or do you do any sort of voice-recognition or lexical analysis? Like looking for certain trigger words?

JF: We tend not to do that because it's so context specific. There can be some advantaghes to pick out some sort of tone based on word choice, but we haven't found it to be that practical and useful yet. But it's an avenue of exploration and we have some partners that work with us on that kind of thing. We're audio only and have a collaboration with USC where we're workign on combining audio with visual cues, to work with teleconference. How can you augment the social signalling by looking at movmeent and expressions, etc.?

JL: Now that is something you can do with your phone app, right? I was reading you also offer phone software. Does that leverage other signals?

JF: We're kinda doing two sorts of thinking. One is modeling and predicting human behavior over short time periods, over a 10-30 minutes conversation. The other thing is looking at longer time periods, over weeks. To look at that kind of time period we've been developing a platform that someone can, if they choose, download onto their smartphone, and what that will do is provide contuinuous monitoring of activity and sociability, and mood and various other things, all picked up throgh contnuous monitoring of smartphone sensors. There are really two different platforms. I can see them converging at some point, but for the moment they're really two different platforms.

JL: And are these available for iPhone and android, or..?

JF: Smartphone is running on Android right now, but it's not available to the general public.

JL: That was actually my next question. I can see this being an interesting platform. You can immediately think of other applications of this biofeedback. The focus now is on health behavior, but I imagine there are other domains, too.

JF: I think there are many broad applications, and in time I think they'll all be addressed, but we must start somewhere. We're starting with problems that we think are really important to solve: distress, but also engagement with organizations. But I'd love to see the platforms broadly adopted, and I don't think it will take us too long to get to that point.

JL: At this point then I can see that your two main customer groups seem to be health initiatives - so people who are monitoring people to make sure they're taking their medication or simialr - and your other main group looks to be customer service, call center type places.

JF: I think that's fair. From an industry perspective there are two customers, the governemnt, where we're mainly working with recently returned soldiers with psychological distress. And the other group is really enterprise. You can think of that as customer service, customer engagement, and that's really helping them have more successful interactions.

JL: And with the busineses it's almost always in the context of a customer service call center?

JF: On the enterprise side it can either be a call center type environment, or else a teleconference environment, depending on the type of interaction. But call center is by far the most common kind of deployment model.

JL: The two main domains that would be cool to take these to, beyond where you are, would be games - as a playtesting tool, actually haveing a good metric of if [the player] is enjoying [the game], modulating game difficulty, etc. - and education, as a tool in the classroom. Is this something you have thought about?

JF: We've explored the adaptive games the least, but it's an interesting idea. On the eudcation side, we don't have any specific projects, but we've met with the Gates foundation a handful of times, as they're sponsoring a lot of work in this area. We're interested, but nothing is happening right now.

JL: we've hit on most of the big questions, but maybe you can tell me a bit more about "reality mining technology" - can you talk briefly about what this is and how your company fits in?

JF: the central concept is to use technology to observe behavior, and then make inferences based on behavior rather than just using self report and survey question. With sensors...we can really use that information to create a better world. Sandy Pentland [Cogito co-founder] is one of the most powerful data scientists in the world. A number of the conceptual frameworks that we've leveraged at cogito came from his group.

JL: Would you differntiate what you call reality mining from the "quantified self" movement? And if so, how?

JF: I think the quantified self is more about an individual understanding his behavior, and I think it's not necesarily behavior; it's anything that can be quantified. But when I think about reality mining, it can be groups of people and how they move and react, on more of a macro scale. I also think the sets of information tend to be almost exlusively behaviorally focused. For example, a physiological measure like skin conductance is not something we focus on.

So there you have it. If you're interested in learning more about Cogito, be sure to check out their website, and for more information on reality mining, take a look at the MIT Media Lab's page on the topic.

Reality Mining at Cogito by Jared Lorince, unless otherwise expressly stated, is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License.

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>