The Case for Metadata Retention

The government's plan to force telecommunications providers to retain a set of metadata for every person has privacy advocates up in arms and police and security agencies telling us that this legislation is essential for fight crime in the 21st century.

So, who is right?

In this two part series we look at both sides of the argument.

Tim Morris is an Assistant Commissioner of the Australian Federal Police. With over 30 years of experience investigating and solving crimes, he can speak from a position of authority on what it takes to catch and prosecute criminals.

Morris was one of the keynote speakers at the Tech Leaders forum held in Sydney in February 2015. He discussed the reasons the proposed data retention legislation was necessary for the AFP to pursue criminals and resolve crimes.

He opened his presentation saying "metadata continues to play a central role in most successful crime investigations".

He cited statistics such as 92% of counter-terrorism, 87% of child protection and 79% of organised crime had metadata a central element of the investigations. However, he noted that the data retention processes in different industries and companies were inconsistent and that this hampered the police's efforts.

"The value of telecommunications data in protecting public safety is, from an AFP perspective, indisputable," he said.

In Morris' view we are now at a tipping point where the need to retain metadata is required to fight modern criminal activity.

Read more: Centrify expands identity management to protect big-data honeypots

According to Morris, the proposed bill does not add any new powers or access to police agencies. "The bill simply obliges companies to retain a limited dataset for a minimum of two years. The limited range of metadata is limited to information about a communication; the who, the where and the when. Not the content or the substance or the substance of a communication," he said.

As examples of what the AFP mean by metadata Morris noted that the phone numbers and duration of calls were included in the data set but not the content. Similarly, with email it would included the email addresses of the senders and recipients but not subject lines or body text.

"The AFP wants industry to also retain the IP address allocated to a user's device, which is a critical piece of data for law enforcement and security agencies," he added.

Morris also took the opportunity during his speech to address some of the concerns raised through the media and other channels. For example, there are some parties that say the proposed legislation places the entire population under mass surveillance.

"Data sitting on a carrier's network is not mass surveillance and we are not speculatively spying on people," he said. He also highlighted that the AFP's role was to investigate people suspected of having "committed individual criminal acts" and to not indiscriminately trawl through data hoping to find potential criminals.

"Agencies can only access this data in limited circumstances, on a case by case basis, where it's reasonably necessary for a lawful purpose," he said.

Access to the retained metadata would require an investigating officer to receive sign-off from a senior, commissioned officer - individual officers won’t have unfettered access to the metadata. And the existing warrant process, which Morris noted was rightly strenuous and under judicial oversight, would still be needed in order to access the actual content of any communication.

Requests for the metadata would be subject to audit by the Ombudsman, who would have increased rights of inspection, as well as ministerial oversight, Senate Estimates, parliamentary committee enquiries and other bodies.

Under the current metadata access regime, Morris said 54000 requests for data were made against about 44 million connected devices.

According to Morris, the new legislation represents a tightening in which agencies will have access to the data with only those who have a "clear operational investigative need".

The currently definition of agencies who can ask for access will be replaced by a short list of key agencies who will specifically have access with new agencies only added if they satisfy specific criteria and are approved by the Attorney-General.

On the question of cost, Morris told the audience that the proposed metadata retention regime is being imposed "for national interest reasons and, as such, the government is prepared to pay a reasonable proportion of the upfront cost associated with the data retention scheme".

Read more: The week in security: Data retention looms, Superfish gutted

Currently, the AFP pays telecommunications providers about $1.5M for access to metadata so that the cost of access to this data isn’t passed on to customers.

Addressing the issue of why metadata would need to be retained by telecommunications providers for a minimum of two years, Morris said, based on experience, there was no correlation between the length of time data was held and its usefulness. He also noted that the AFP only requested data it knew was available so, in many cases, data more than a year old was rarely requested as the AFP knew it would not be available so concerns raised about the two year period being in excess of current requirements were based on an incorrect assumption.

Many crimes are only identified many months, or in some case years, after they have occurred.

Morris highlighted the importance of the various elements of the proposed legislation by using three different cases to illustrate the AFP's position. Operation Pendennis, Operation Inca and Operation Drakensberg. These terrorism, organized crime and child exploitation cases were successful cases where metadata was used to successfully find and prosecute criminals.

He also noted, in Operation Drakensberg which was a multi-year international investigation that originated in the UK and it took two years for the UK police to send the case to the AFP. The metadata required to investigate 41% of the Australian offenders was not available as carriers did not retain the data for a sufficient period. As a result, none of those 41% of potential offenders were investigated.

We asked Morris whether the AFP would be selective in the crimes it would investigate using metadata and whether software and media piracy was a crime the AFP would investigate using metadata.

"One of the threshold tests in the Act is 'is it reasonably necessary?'. Let's say it's a trivial or minor offence. You've still got to pass that threshold test. Is the intrusion to get someone's data reasonably necessary? The AFP is not interested in someone sitting in their lounge room torrenting Game of Thrones. We're not going to have a taskforce come out to get you based on metadata that we've collected".

In his concluding comments, Morris noted that the AFP was committed to protecting the privacy of individuals but that "the AFP can not support the right of anonymity, especially when it becomes related to unlawful activity".

Image from

This article is brought to you by Enex TestLab, content directors for CSO Australia.

Upcoming IT Security Events

March 3rd, March 5th, March 9th 2015

Join CSO for the day@#csoperspectives and hear from @kimzetter @LeviathanSec

3 International Keynote speakers, 36 Key IT Security Industry Speaker, 21 Exhibitors, Security Analysts and many more.. Register today

Dont miss one of the biggest IT Security events in ANZ (registration is free, but seats are limited)

Tags TelecommunicationsAustralian Federal PoliceAFPOmbudsmanTim Morristech leadersCSO Australiametadata retentionOperation Drakensbergdataset

Show Comments