BlogUncategorized

Productivity vs Privacy: The Debate on Microsoft’s Total Recall

Microsoft has unveiled its latest AI innovation. Copilot+PC is a PC-based generative AI solution that will allow the Windows user to take advantage of AI locally without having to access the cloud. You’ll be able to ask  natural language questions, generate AI art and much more. Microsoft calls the Copilot+PC the fastest, most intelligent Windows PC ever built.

Instant Recall

As part of this system it introduced the Recall program. An AI powered tool that Microsoft claims will solve the problem of users forgetting what they did on their PCs. The system will allow them to search for anything they have seen on their screens even if they did not bookmark or save the content.

To do this, the Recall system will automatically log everything a user does, including app activity, browsing sites and more. It will make transcriptions of live meetings and videos, take screenshots of the user’s display every five seconds and recognize the objects and text in the images. All this information will be aggregated to a local Large Language Model (LLM) and made searchable to the user for complete recall.

Your Recall Timeline, Visualized

Your recall timeline, visualized

If you’re thinking this sounds like spyware then you are not alone. The reaction has been swift with BBC calling it a ‘privacy nightmare’, cybersecurity researchers being appalled and comparisons being made to that creepy eye gismo episode in the suspense series Black Mirror. The UK has already opened an investigation into the system, citing confidentiality and consent worries. If somebody gained access to your account they could very accurately and easily trace back your activity timeline and replay the exact things you were doing, searching or viewing on your PC, all with related screenshots.

Forensics

Looking at it objectively, this is nothing new. Windows has always recorded user activity. One group of people who may like this is digital forensics investigators, such as in our forensic team. In deep dive investigation, they are often asked to create a timeline of user activity to uncover what a user did. They look in a number of locations to create these timelines and the process is technically challenging. Recall may just make this easier. PC users may not be aware of how much activity is already recorded.

The Windows registry, for example, is a database of options and configurations in the heart of the Windows OS. One artefact, called a ShellBag, stores the name, path, windows coordinates, and time of folders that were opened on the system. The record persists even if the folder is deleted.

There are also numerous “Most Recently Used” lists that record the files that a user opened, software that they have run, websites they have visited and in which order the actions occurred.

Windows Volume Shadow Copies, also known as Volume Snapshot Service (VSS) or Shadow Copy, is a Windows feature that creates backup copies of files and volumes, even when they are in use. Volume Shadow Copies automatically backs up files and can reveal deleted files, recent changes and other information a user thought were long gone.

Recently, the Microsoft Edge browser included an option to store screenshots of your browsing history in addition to the webpage address record that it already logs. This feature is off by default.

Forensic investigators take all this information and create timelines of user behavior. The Recall system may just make it easier for them to do so.

Microsoft’s Response

Microsoft, to its credit, seems to be aware of the privacy concerns and has tried to allay fears.  All information is stored locally in an encrypted format, the GenAI model would be based on the local dataset and would not transfer any data out to Microsoft servers. Users will also be able to turn the system off or pause it if needed. They can exclude logging of specific apps and the private browsing activity of supported web browsers. It’s also still undergoing further testing so Microsoft may change it further.

Nevertheless, it seems to be an opt-out system rather than opt-in, which, in a controversial system like this is not desirable. Recall will record your banking details and anything else you may be doing in normal browsing modes unless you remember to turn it off.

The system comes with a significant hardware cost. Copilot+PC systems need to be powered by Qualcomm’s Snapdragon X Elite chips, which include the necessary neural processing unit (NPU), and at least 25Gb of space for Recall storage (about 3 months’ worth of screenshots).  If you really don’t like Recall, just avoid PCs with this type of setup when shopping for new hardware.

Microsoft has also confirmed that the system won’t be available on older systems. So, this is not a concern for organizations using existing Windows PCs.

The ability to scan and rewind your timeline may be a fun concept and useful in very narrow circumstances but it may not be worth the privacy and confidentiality questions that it raises. The screenshot feature in particular is coming under great scrutiny.

Our digital forensics investigators may have a new source of information in the future, but I would not be surprised if this system is severely curtailed once it hits the mainstream users’ PCs.

ProSearchers continually watch new tech developments analyzing the potentail impact on data and risk management

For more on the AI features in CoPilot software, read Ryan Hemmel’s post on Using AI Every Day.

 

Filed under:

BlogUncategorized
Damir Kahvedžić

Damir Kahvedžić

Damir Kahvedžić is a technology expert specializing in providing clients with technical assistance in eDiscovery and Forensics cases. He has a PhD in Cybercrime and Digital Forensics Investigations from the Centre for Cybercrime Investigation in UCD and holds a first-class Honours B.Sc in Computer Science. Experienced in the use of industry leading software, such as Relativity, EnCase, NUIX, Cellebrite, Clearwell, and Brainspace, Damir is also a PRINCE2 and PECB ISO 21500 qualified project manager. Damir has published both academic and technical papers at several international conferences and journals including the European Academy of Law, Digital Forensic Research Workshop (DFRWS), Journal of Digital Forensics and Law amongst others.