Take a quick scroll through the photo album on my iPhone and you’ll discover I have two kids, play the occasional round of golf and recently took a trip to meet Mickey and Minnie in Paris.

There are a few screenshots of wallpaper samples too, which may give you an idea that we recently did some decorating at home.

Like many people, I would imagine, the most used feature on my phone is undoubtedly my camera – a quick scenic snap for Instagram that I’ll filter later or a group shot for the family album (well, the online one anyway). Photos by far take up the most storage on my handset, so much so that I often have to start deleting some before I can take anymore.

That’s a marketing manager’s goldmine according to Pixoneye, the London and Israel-based start-up that believes it has come up with the most accurate way of personalising content to customers.

The albums on our phones paint a detailed picture of who we are, what we’re into and what’s happening in our lives. Our social media posts are what we choose to share with the world – often heavily filtered and edited – whereas a phone is an untapped resource offering a true image. What if brands could use that information to find out more about their customers?

That was Ofri Ben-Porat’s goal when he set up Pixoneye after working as a marketing specialist with the Israeli government. His puzzle there was in marketing the country to the two very different groups of tourists it drew in – evangelical visitors who travelled to the country on pilgrimage and the LGBT community, drawn to Tel Aviv because of its liberal outlook. The challenge was to find a pure data set to distinguish the most relevant information to market to each particular tourist.

While chatting to a friend who had helped build the facial recognition algorithm for the Samsung Galaxy, he and co-founder Nadav Tal-Israel hit upon the idea of using imagery to understand the end user with the help of AI.

“One thing we all have in common is that the smartphone is a portal to our lives and photos have become a social currency,” says Pixoneye’s chief commercial officer Richard Jones (below).

Richard Jones

“People who go to gigs tend to watch through their phones and if I was to send a message to my team I’d probably send a photo to go with it rather than just a text. We’re creating an incredibly detailed trail of who we are as people on our phones.”

The result has been a software development kit (SDK), a program that operates in the background of a brand’s own app, which gleans information about the customer or user via their photo album. The SDK analyses the images to come up with feature vectors, presented in the form of a graph. The software can reveal up to 150 detailed characteristics about a person, enabling brands to produce first-party data that belongs to them rather than having to buy in third-party data. They then choose how to use that data to shape the in-app experience.

“The businesses using the SDK have a highly visual dashboard that breaks down all the different characteristics, so they know whether the customer is male or female, whether or not they have children and pets, their propensity to travel and where they live, for example,” says Jones.

A brand could then personalise content or adverts on the app: for example optimising a homepage or tailoring push notifications to the person. Building a detailed picture of its customer base could also generate more custom by targeting people from a similar demographic.

“One of the most exciting features is not just about understanding who the customers are, it’s about understanding their intent,” Jones says. “I started to take images of furniture and floor plans and share them with my other half recently, a clear indication that we were about to move home. Knowing that such a significant life event is coming up gives an incredibly unique understanding of that end user.

“We’re creating characteristics based on client feedback, but also finding a lot of traction around the core characteristics we’ve identified. We’re an evolving organisation and we can train our algorithms to look for different things.”

Pixoneye

Clients in financial services, retail and the insurance sector are offering positive feedback and there is huge potential for utility businesses. “Within the utilities sector, 25 per cent of end users leave every year because it’s a price-driven purchase,” Jones says. “Using our technology, a utility business could be alerted to when a customer is about to move home so they can change their dialogue accordingly.”

Then there is the potential for credit companies to use the technology to help build a risk profile of a borrower – a use Pixoneye anticipates in developing countries. “Some countries don’t have the data trail over somebody’s lifetime and that could mean the difference between them not getting their credit extended,” Jones says. “This kind of personalisation could be a game-changer. You could find out whether somebody is married or has a job [for example]”.

Though some of the examples are speculative, there have been some proven results. Nestle wanted to target its pet food ads specifically to cat owners or dog owners and called on Pixoneye to help. The SDK was integrated into the Daily Mail app for one month and analysed the photo albums of 1,000 users to gain feature vectors. Using this information, Pixoneye was able to determine which users were cat owners and which were dog owners, and both groups received Nestle Purina ads specific to their pets. A test using standard marketing methods ran alongside this. The test with Pixoneye installed increased click-through to purchase by ten times, from four in 1,000 to 40 in 1,000, a huge increase in engagement and revenue.

A similar study with Tesco’s shopping app used the SDK to analyse the photo galleries of 10,000 shoppers over a month. Ninety per cent of the feature vectors were matched to the corresponding shopping carts of those customers. The remaining ten per cent were used to predict future shopping cart contents, with 92 per cent accuracy.

It all sounds like good news for businesses, but what about customers? Could the thought of an app scanning your personal photographs while you use it be a step too far for some?

“The user can opt out and we’re very conscious about the importance of engaging effectively with them,” says Jones. “We’re in a time when privacy is a significant concern and people are learning more about how their data is used.” To stop your album being used in such a way, you’d have to go into settings and prevent access to your photos by the app, although you have no way of knowing if that particular app uses Pixoneye. The other important thing to note, Jones says, is that your photos are not uploaded to the cloud as the SDK runs within the app. The feature vectors also wouldn’t reveal exactly what was on your photos, but merely a ‘profile’.

Jones believes this can only be a positive for the customer. For now, Pixoneye is broadening its product, improving delivery of the results to the customer and gaining traction in a variety of markets. Research and development is carried out from Tel Aviv, while the commercial side is run from London. He admits they have “other innovative tech plans up our sleeves as well”.

“The key thing is improving the experience of an app and enriching users’ lives with better content and products,” he says. “From the perspective of the end user, it’s very secure.

“I liken it to having a true personalised shopper in your pocket, having the best shopping experience you can have.”

Enjoy extra content and interactive videos in the interactive digital magazine below

E-edition cover