The Hong Kong government has confirmed the existence of a facial recognition feature in the source code for its contact tracing mobile app LeaveHomeSafe, but has claimed that it has never been used.
In response to a FactWire report about the existence of a facial detection feature in the app’s source code, the Office of the Government Chief Information Officer (OGCIO) said that it had asked the app’s developer to look into removing the facial recognition source code without affecting the “normal operation” of the application.
LeaveHomeSafe was developed by Cherrypicks Limited, a subsidiary of the Chinese online gaming company NetDragon Websoft Holdings Limited, and has been available for download since November 2020. It supports the iOS, Android and Huawei operating systems.
Members of the public can use the application to record when they enter and leave certain venues by scanning the venues’ QR codes.
FactWire examined the source code of the latest Android release of the LeaveHomeSafe app – 3.2.0 – along with six previous versions by converting files downloaded from the app’s official website into human-readable java source files.
The converted source file comprises some 20 folders. One of those, named “org,” contains a subfolder named “reactnative,” which further comprises three folders titled “facedetector,” “camera,” and “maskedview” respectively.
React Native is an open-source software framework for developers to incorporate into their applications. These folder names and the code found in the “camera” folder suggest that the LeaveHomeSafe app adopted code from a React Native codebase called “react-native-camera,” which drives mobile device cameras.
However, most of the modules, functions and classes in the java code had been renamed, leaving alphanumeric code in their place using a process called code obfuscation. The process is commonly used to protect applications’ copyright and to stop them from being cracked. Three of the four java files under the “facedetector” folder of the LeaveHomeSafe app had been obfuscated in this way.
To understand the use of this obfuscated code in the LeaveHomeSafe app, FactWire compared it to the open-source React Native code and found that the structure of the obfuscated code matched that of the codebase known as “react-native-camera-master.”
One of the java files, originally known as “FaceDetectorUtils.java” but renamed “a.java” in LeaveHomeSafe, could be used to detect the position of a person’s mouth, nose, left and right cheeks, eyes, ears and earlobes. It is also able to detect the degree of a head tilt and calculate the probability that someone is smiling or has both eyes open.
The facial detection module can be found in all versions of the LeaveHomeSafe app that FactWire examined, suggesting that it has existed since the app’s early launch in 2020. This is, however, not mentioned on the app’s official website, nor in mobile app store descriptions.
Technical specifications published on the LeaveHomeSafe website state that the React Native framework was used to develop the app. Since the same framework and codebase can be used to write Android and iOS apps, the codebase for the Android and iOS versions of LeaveHomeSafe are thought to be largely similar.
FactWire has examined the code of the Android releases, but not the application built for iOS. Version 3.2.1 was released on April 30 for iOS, but the latest version for Android remains 3.2.0 at the time of writing.
Faces, barcodes, QR codes
Marc Rousavy, who personally maintains the “react-native-camera” module and whose app development and tech consultancy firm Margelo was responsible for writing React Native’s core, confirmed to FactWire after evaluating LeaveHomeSafe’s code that a facial detection module can be found in the app, since it employs the “react-native-camera” module.
However, it is difficult to tell whether the app’s facial detection module is active because the code that contains information on how the app runs has been encoded.
Despite this, Rousavy found that the identifier of a function named “onFaceDetected” was passed to another function within the 3.1.0 version of the app, which could mean that something is written to the device’s disk when a face is detected, although what that could be remains unclear.
FactWire found that that same part of the latest version of the app was slightly modified. The “onFaceDetected” function may be reading some kind of information instead when the function itself is called or when its identifier is passed to another function.
Leo To, a software engineer who has worked for the government as a contractor, believed that the reason a facial detection module can be found in the LeaveHomeSafe app was that it simply originally existed in the ready-made “react-native-camera” module, and was not removed by the app developer when adopting the module. Apart from detecting human faces, the same module can be used to detect barcodes and QR codes as well, he explained.
In To’s opinion, however, unused modules should be removed from the application’s source code, as they might create operational problems. “It is definitely possible to import the entire module and remove just the facial detection part,” he said. “The developers from [the government-contracted app developer] Cherrypicks have, of course, the technical ability to do it. It is just a question of whether they decide to do it.”
He also suspected that the OGCIO may not have known about the code, saying that in his experience, the government did not inspect the source code, despite requiring contractors to submit it. “They would only ask you to fix certain problems that they encounter when testing the app,” To said.
To said there was no need for the app to ask for permission to activate the facial detection feature as the app was able to record a video using the camera, which it already had permission to use to read QR codes and car licence plates. There was no indication as to how the video would be used in the system’s backend, To said.
FactWire tested the LeaveHomeSafe app by entering debug mode using Android Studio and a smartphone and learned that the app currently only uses the rear camera of the device, rather than the front-facing camera, which would be much more likely to capture the user’s face. The probability of the user’s face being detected is thus very low.
In a response to an enquiry from FactWire, the OGCIO confirmed that Cherrypicks used the “react-native-camera” module to allow the app to scan taxi licence plates and the QR codes displayed at certain venues and on vaccination records.
It admitted that the module includes a facial detection feature, but said that it had not been aware of it before receiving FactWire’s enquiry and that Cherrypicks had never activated it, adding that such a feature was unnecessary for the operation of the app.
The government said it had immediately asked Cherrypicks to “study the feasibility of removing the facial detection module while ensuring the app’s normal operation would not be affected,” in order to eliminate unnecessary public concern.
According to the government’s contract with Cherrypicks, the LeaveHomeSafe app and its source code are the intellectual property of the government. However, the government does not own the source code that the app draws from, such as that already used in Cherrypicks’ previous products and ready-made commercial or open-source codebases and modules.
Addressing privacy concerns, the OGCIO said that there was a need for the app to obtain certain information from the mobile device including the app’s version, its operating system, the device model, Android API level, and to check whether there was a passcode or biometric verification set up in the case of importing a vaccination record.
“We treat the public’s privacy concerns seriously by consulting the Office of the Privacy Commissioner for Personal Data every time the LeaveHomeSafe app is updated with new features, to ensure that it complies with the Personal Data (Privacy) Ordinance,” read the reply.
The government emphasised that the app was “safe and reliable” by stressing that its privacy impact and security risk were assessed and audited by an independent third party. However, neither the security risk report not the privacy impact report, which can be found online, mention inspecting the idle modules of the app, despite the privacy impact assessment stating that its aim was “to identify and address any data privacy implications/issues.”
Cherrypicks Limited did not respond to FactWire’s inquiry, only referring the questions to the OGCIO.
According to a written OGCIO reply to a Legislative Council inquiry on April 13, the LeaveHomeSafe app has cost an estimated HK$8.6 million thus far, of which about HK$3.6 million went on maintenance and upgrades of the application and its backend systems. The remaining HK$5 million went on support and operation, including staff salaries.
The Secretary for Innovation and Technology Alfred Sit announced on April 13 that version 3.2.0 was expected to be released in early May, with a new feature to import recovery records. It was released on April 18, five days following the announcement.