Private Eyes: Are Retailers Watching Our Every Move?

  • Share
  • Read Later
Getty Images

Traditional brick-and-mortar retailers don’t have a lot going for them these days. On the one hand, high unemployment and stagnant wage growth is sapping the purchasing power of the average consumer, and on the other, the rise of e-commerce is giving those consumers more information and choices than they’ve ever had before, making competition all the more fierce. As I detailed in a recent article, store managers have been fighting back by trying to re-create in physical stores the sort of analytics available to e-commerce firms. Firms like RetailNext can use security-camera systems to give retailers a tremendous amount of information about customer behavior in stores, allowing retailers to finely tune staffing levels and product placement. Other firms, like Euclid Analytics, provide the same information by identifying customer smart-phone wi-fi signals.

And while these analytics firms provide invaluable intelligence to retailers that is enabling them to improve their operations and boost profits, privacy advocates are worried about how far companies will take these technologies. After all, it’s one thing for a retailer to have a general idea of how many people are in the store and how, in the aggregate, consumers are interacting with the store; but it’s another thing entirely for a retailer to be able to identify a customer individually and tailor pricing and service based on his in-store behavior and financial history.

(MORE: Future of Retail: How Companies Can Employ Big Data to Create a Better Shopping Experience)

Pam Dixon, executive director of the World Privacy Forum, says that although most of the focus in the media has been on how companies are tracking us through Internet browsers and smart phones, there is actually more danger of invasions of privacy occurring in physical retail outlets, mostly because consumers are unaware of the extent to which they are being tracked. “This is an entire business model that has sprung up that I think maybe three people in the entire country know about outside the industry,” she says.

And though analytics firms and retailers claim they aren’t using technology to personally identify shoppers or pair that information with financial histories, it is very much possible to do so. In 2010, the Association of Marketing in Retail produced a voluntary code of conduct for marketers and retailers to use as a guide in their tracking and marketing efforts. The code outlines the various tracking capabilities available and rates them on a scale from low risk to high risk. According to the code of conduct, a low-risk tracking method would include “infrared or laser or laser beam motion detectors” that can give retailers an idea of how many people are in a store and where they are traveling but “are not able to track or record individual consumer paths.” The high-risk end of the spectrum includes methods that allow retailers to individually track consumers by recognizing a smart phone wi-fi signal or through interpreting visual data from facial-recognition technology.

That kind of tracking is, according to Dixon, unethical and contrary to shoppers’ expectation of privacy. “Legally, stores have the right to put up security cameras, but the consumer expectation of privacy is being circumvented here,” she says. “Because when a consumer looks into that camera, they expect it’s being used for security, not marketing purposes.”

According to Mark Eichorn of the Division of Privacy and Identity Protection at the Federal Trade Commission, the FTC has been monitoring this type of consumer tracking but hasn’t found that firms are using facial-recognition software to create individual profiles of customers. Last December, the FTC held a workshop on facial-recognition technology in the retail space; Eichorn says, “We didn’t hear from anybody saying they have a business model where people were using surveillance cameras to identify people.” Instead, businesses are using the software to get aggregate information on the number of people or the general age or gender makeup of people in a retail setting. “We would be very concerned about the use of cameras to identify previously anonymous people,” he says.

(POLL: Is It O.K. for Retailers to Track Customers’ Shopping Behavior While in Stores?)

But the problem, according to Dixon, is that we can’t be sure how retailers are using this technology, because they’re not required to disclose those details. And even if retailers were using facial-recognition software to identify individual shoppers and combining that information with other data like financial histories, there is nothing necessarily illegal about doing so. The FTC can penalize companies that engage in deceptive or unfair conduct, but the agency has never brought any enforcement action against retailers for this type of tracking, so it’s unclear how courts would interpret these statutes as they relate to in-store tracking.

Bill Gerba, CEO of the digital-signage company WireSpring Technologies, helped the Association of Marketing in Retail create its code of conduct. While his company doesn’t facilitate the kind of tracking technologies described above, he has been interested in the move toward tracking in retail and has written extensively on the subject. He thinks consumers should be informed about these marketing tactics but believes most retailers are too wary of the public backlash to use facial-recognition software to individually identify their customers. He also believes most companies are relying on smart-phone apps to track customers’ locations and push relevant marketing material. This method of tracking allows retailers to offer an explicit opt-in prompt when a user downloads the app, mitigating many of the privacy concerns. “Retailers see this as a nice compromise, and it prevents the customer from being bombarded by digital signage or greeters coming up to talk with them by name. It’s only existing in their mobile phone, and if they don’t like it, they can unsubscribe from the app,” Gerba says.

But even if most retailers are shying away from using facial-recognition software to its fullest potential, the chance that some are, without our knowledge, grows greater as the technology becomes more sophisticated and affordable. And because retailers are implementing this type of tracking on their private property, a consumer’s legal rights to privacy are very limited. But privacy advocates say that even if this kind of tracking can’t yet be legally stopped, consumers should at least be made aware of what’s going on. Says Dixon: “I think it’s absolutely crucial for these companies to come clean with the public and disclose what is happening.”

MOREGoogle Fiber Issues Public Challenge: Get Up to Speed!

How do you feel about retailers’ efforts to track consumer behavior? Weigh in through our online poll, and be sure to join us on Friday, Sept. 21, at 12:30 p.m. at business.time.com for a live Google+ Hangout discussion of privacy and in-store tracking.

20 comments
Video Surveillance
Video Surveillance

At VideoSurveillance.com (http://www.VideoSurveillance.c...), we’ve seen an increasing trend across all industries, but especially in the retail sector, where customers want to leverage their networked HD-quality security cameras for additional, non-security applications.

There are several interesting emerging video analytics

solutions which aggregate customer data (customer flow/heat maps, dwell time,

etc) rather than analyze individual behavior.  These provide retailers

actionable, accurate insights on an ongoing basis, rather than one-off in-store

studies compiled by hand.

There is a large opportunity for individual-level retail analytics in the future.  However, at this point (and for the foreseeable future), real-time facial recognition as described in the article is pure fantasy. Implementing these solutions as part of an “Opt-in” program tied to a Smartphone app makes the most business sense for retailers, both from a data accuracy and customer service perspective.  From a privacy perspective it’s identical to long-standing ‘rewards card’ programs in place at most supermarkets.

Manmohan Manu
Manmohan Manu

Good information. I work for McGladrey and there’s a white paper on the website about this very topic readers would find useful. http://bit.ly/JDHmUU It describes several issues impacting retail sales, closures and acquisitions.

Manmohan Manu
Manmohan Manu

Good information. I work for McGladrey and there’s a white paper on the website about this very topic readers would find useful. http://bit.ly/JDHmUU It describes several issues impacting retail sales, closures and acquisitions.

F.c. Mansfield
F.c. Mansfield

well over half of all loss due to theft is done by employees. those cameras are placed on the one or two high end/high margin aisles, but the vast majority of surveillance is to document the work force. next time you are in a store with a lot of cameras, you will find they are pointed at registers and exits, with more cameras in storeroom areas. retailers are not as progressive as they pretend to be, they generally follow the lead of someone else who had a good idea.

DJC27
DJC27

 Also, the majority of "in-store" cameras not pointed at registers is used for slip-and-fall fraud. Ah, the beauty of the American legal system!

max4374
max4374

Last year while visiting San Diego, my wife went into a cosmetics store, which is a national chain.  She bought one item, payed cash, did not show any I.D., but starting one week later she was inundated with email adds from that company. We always wondered how they knew who she was, and her email address.

Tsuki
Tsuki

some retail chains like Sephora ask for your email address when  you make the purchase even if you pay with cash. Either that or she signed up for a rewards program.

max4374
max4374

 No on either, that's why she pays cash

KarimRonaldo
KarimRonaldo

Frank explained I'm in shock that a person able to make $9720 in 4 weeks on the network. have you seen this(Click on menu Home)

Tracy B. Adamson
Tracy B. Adamson

The FTC can penalize companies that engage in deceptive or unfair conduct, but the agency has never brought any enforcement action against retailers for this type of tracking, so it’s unclear how courts would interpret these statutes as they relate to in-store tracking. http://goo.gl/zWgLS

DJC27
DJC27

Perhaps because she used your personal computer to go on their website? Even if, magically, they identified her through video surveillance, how would her face be linked to an email address? Gosh some people are gullible...

max4374
max4374

 Facebook has face recognition.  Even with my privacy settings full on, friends and their friends can still name me, even if they do not tag me without my approval.  You can take any photo and as long as someone posted it somewhere on the net, you can find it with Google Images or other programs.  Finding who it is, is relatively easy.  My question was how do they find her email associated with her image?  And no, she didn't register on their website.

Please, do not accuse others that are just commenting and simply want to know.

DJC27
DJC27

Max, I do apologize if I sounded condescending earlier, that wasn't my objective. It was more reactionary than anything.

Programs such as Facebook and Picassa are able to recognize faces. One thing to remember, however, is that most pictures are taken by cameras that today have at minimum 8 megapixel of resolution (so around 3264 pixels by 2448 pixels) and subjects are standing within a few feet of the camera, looking directly at it. Security cameras, on the other hand, have for the most part a maximum of 640 pixels by 480 pixels of resolution (called VGA, the resolution you had on your old TV set basically), so significantly less detail (about 30 times less than an 8 megapixel point and shoot camera), are located 30-50 feet from the subjects, at skewed angles with people looking in random directions. More and more manufacturers are introducing megapixel security cameras (2mp now becoming increasingly popular), but the retail market specifically has been slow to catch on. This is actually the main reason why facial recognition is simply not possible in the security world yet.

My point is, your wife's image had nothing to do with how they got her email address, as the two are completely unrelated. She could've worn a mask, hat and sunglasses that day, and still would've received email spam. What most likely happened is that she would've visited their website (no need to sign up for anything, they can track you with cookies and look up your email address if it's saved in your computer's cache), although this practice is now somewhat illegal, or she did sign up to something, either in store or online. Another possibility is that the chain in question are extremely proactive, and tracked her phone ID through the wifi signature which contained her email address, although I am not 100% certain on the feasibility and legality of such a thing.

Again, one thing is certain, the store's security cameras had nothing to do with it.

rjs0
rjs0

"According to Mark Eichorn of the Division of Privacy and Identity Protection at the Federal Trade Comission, the FTC has been monitoring this type of consumer tracking, but hasn’t found that firms are using facial recognition software to create individual profiles of customers.  “We would be very concerned about the use of cameras to identify previously anonymous people,” "

this video clearly shows individuals identified by facial recognition and their buying history logged as they move about the mall, using neuroscience methods to measure brainwave activity in real time, and capturing purchase considerations at the moment they are formed in the brain:

http://marketwatch666.blogspot...

DJC27
DJC27

Unfortunately, the video you linked is bogus; it's simply a demo video used to showcase potential technology. It's quite easy to determine as the cameras used are broadcasting cameras, and not security cameras.

I've been in the security industry for 8 years, and the biggest challenge we face, day in day out, is misinformation and unrealistic customer expectations. CSI is a TV show, and the vast majority of what you see on TV and in movies simply does not exist.

Although it is true that marketing tools are evolving through video surveillance analytics (such as people counting, queue management, slip and fall detection etc), these technologies are barely used by retailers today because they simply do not work well enough, and cost way, way too much. Simpler, older technologies, such as IR motion sensors, are significantly more accurate, and significantly less expensive. As we move forward, these video analytics will definitely become more and more popular. However, facial recognition is not, today, a technology that is even close to being used by mass markets such as retail, airports, etc. Maybe one day? perhaps, but not in the next decade...

A quick trip to any security trade show (ASIS, the 2nd biggest security trade show in NA, was held in Philadelphia just last week) will show that facial recognition does exist. It is sold as a tool to identify a user, based on an existing database, in an extremely well controlled environment, where a user stands within inches of a camera. When use (as it is seldom deployed), it is used as an access control tool. Facial recognition as we see it on TV simply does not exist today. Think of the distances, the angles, the filed of view; if we can barely identify someone captured on camera (think of your evening news or America's Most Wanted), how could a computer algorithm identify that person? Reality is, it looks flashy in a movie, but that is the extent of it.

This article is nothing more than fear mongering. It sounds controversial, it attracts people's attention, but it is as misinformed as the Obama birther debate... The author writes that "retailers are shying away from using facial recognition software to its fullest potential". They are not shying away from it; it simply does not work.

The only valid point made by the author in the article is when he talks about cell phone wifi signatures. This technology exists, and is actually becoming more and more popular, although still in its infancy.

A bit of research on facial recognition would've saved the author a lot of time... And to the general population, if you are really that concerned about mass facial recognition, simply wear sunglasses or a hat... yes, the technology (at its best, as described above) is that unreliable...

DJC27
DJC27

 This video you linked is a demo of what scientists think might be possible. It is in no way, shape or form a real video using real technology. I work in the security industry, for the world's leading video surveillance company, and have been in the industry for close to 10 years. The video you linked, first of all, isn't using security cameras, but rather broadcasting cameras; it was made as a demo video... Might as well link a clip from CSI...

Although it is true that some technologies can be used as marketing tools through video surveillance (people counting, waiting time, time spent in front of a specific display, slip and fall, etc), the actual accuracy of these technologies is still far from what it needs to be in order to make it cost effective. People counting, the most basic of these tools, is barely used in the retail market, because using older, simpler technologies such as IR sensors is significantly more accurate, and significantly more cost effective.

As for the subject of this article, facial recognition, a tiny bit of research by the author would've saved him a lot of time. Facial recognition exists; it is demoed, year after year, at the big security trade shows around the world. It works (ok) in a very controlled environment, where the subject is looking directly at the camera, within a few inches of the lens. For now, it is used mostly as a tool for access control. In no way is facial recognition used, today, as a tool to identify people within a crowd, like what we see on TV and in movies. The technology simply isn't there. A bit of common sense can also support this: think of the angle, the field of view and the distance. Think of the shots you see of a suspect on America's Most Wanted. If a human can barely identify someone they know while looking at security footage, a computer algorithm won't be able to do so either.

Retailers aren't "shying away from using facial recognition software to its fullest potential", they simply don't use it because the technology doesn't exist. The use of cellphone identification is, however, an up and coming technology that exists and works. If the author would've focused on that, the article would've made a lot more sense.

This article, unfortunately, is one of those scare tactic article mainstream media loves to shove down our throats...