Skip to main content

Please be advised that our office will be closed from 5pm – Tuesday, 24 December, and will reopen on Thursday, 2 January 2025.

Privacy
  • On this page

Publication date: 18 June 2014

Key terms

‘Administrative action’ means action taken in response to a match obtained through a data matching program that materially affects any individual or class of individuals, including, but not limited to:

  • any action directly detrimental to an individual, such as reducing a benefit or imposing a penalty
  • the initiation of an investigation which might lead to action directly detrimental to the individual subject to the investigation, and
  • the disclosure of information to a third party, where the disclosure might cause harm (including embarrassment) to the individual to whom the information relates

’Agency’ has the meaning set out in s 6 of the Privacy Act and includes, amongst other things, a Minister or an Australian Government Department.

’ANAO’ means the Australian National Audit Office.

’APPs’ mean the Australian Privacy Principles, set out in Schedule 1 of the Privacy Act.[1]

’Commissioner’ means the Information Commissioner within the meaning of the Australian Information Commissioner Act 2010.[2]

’Database’ means a structured collection of data or records, stored by means of a computer in a manner that facilitates retrieval. A data set is a subset of a database.

’Data matching’ means the bringing together of at least two data sets that contain personal information, and that come from different sources, and the comparison of those data sets with the intention of producing a match.

’Data matching cycle’ means the completion of all the steps and processes necessary to generate a match, within a specific timeframe.

’Data matching program’ means the conduct of data matching to assist one or more agencies to achieve a specific objective. A data matching program may involve more than one data matching cycle.

’Data set’ means a discrete, ordered collection of data. A data set may be sourced from a database, and may be defined by specific criteria — for example, the receipt of a certain benefit within a given period.

’Entity’ means an agency, an organisation, or a State or Territory authority.

’FOI Act’ means the Freedom of Information Act 1982 (Cth).[3]

’Guidelines’ means this document, including Guidelines 1–13 set out in this document.

’Match’ means a result produced by data matching, including a meaningful discrepancy, in relation to which administrative action may be taken by the matching agency or source entity.

’Matching agency’ means, in relation to a data matching program, the agency whose information technology facilities or resources are used to conduct the data match comparison.

’OAIC’ means the Office of the Australian Information Commissioner.

’Organisation’ has the meaning set out in s 6C of the Privacy Act and, in general, includes all businesses and non government organisations with an annual turnover of more than $3 million, all health service providers and a limited range of small businesses (see ss 6D and 6E of the Privacy Act).

’Personal information’ has the meaning as set out in s 6 of the Privacy Act:

means information or an opinion about an identified individual, or an individual who is reasonably identifiable:

  1. whether the information or opinion is true or not, and
  2. whether the information or opinion is recorded in a material form or not

’Primary user agency’ means, where a data matching program involves more than one agency, the agency that makes the most use of the results of a data matching program.

Usually the primary user agency will also be the matching agency, but there may be data matching programs where the matching agency does not use, or uses only to a limited extent, the results of the data matching program. Where there is more than one agency using the results of a data matching program, those agencies should agree which is the primary user agency.

’Privacy Act’ means the Privacy Act 1988 (Cth).[4]

’Source entity’ means any entity that discloses a data base or data set containing personal information to a matching agency for use in a data matching program.

’Source data’ means the record, including electronic or paper records, from which information (held in a data base or data set) has been provided for use in a data matching program.

’State or Territory authority’ has the meaning set out in s 6C of the Privacy Act.

’Technical standards report’ means a report of the kind described in Appendix B.

’User agency’ means an agency that uses the results of a data matching program.

Background

The purpose of these Guidelines

1 These Guidelines on Data Matching in Australian Government Administration (Guidelines) aim to assist Australian Government agencies to use data matching as an administrative tool in a way that complies with the Australian Privacy Principles (APPs) and the Privacy Act, and is consistent with good privacy practice.

Who should use these Guidelines?

2 This document should be used by agencies that handle personal information and wish to use data matching to determine whether administrative action is warranted.

3 The Guidelines do not generally apply to data matching where Tax File Numbers are used. The Data-matching Program (Assistance and Tax) Act 1990 (Cth)[5] regulates the use of Tax File Numbers in comparing personal information held by the Australian Taxation Office and by certain ’assistance agencies’ including the Department of Human Services (which administers the Centrelink, Child Support Agency, and Medicare Programs)[6] and the Department of Veterans’ Affairs.[7] The OAIC has issued separate mandatory guidelines in respect of the data matching programs authorised by that Act.[8]

4 Further, the Privacy Commissioner has issued mandatory guidelines[9] under s 135AA of the National Health Act 1953 (Cth)[10] that regulate the storage, use, disclosure and linkage of patient claims information collected under the Pharmaceutical Benefits Scheme and the Medicare program. Agencies dealing with Medicare and Pharmaceutical Benefits Scheme information should consider their obligations under those guidelines and any other relevant legislation before matching that information.

5 An agency will not be taken to have acted inconsistently with the Guidelines by virtue of an action that the agency is required or specifically authorised by law to take.

Status of the Guidelines

6 These Guidelines, which are issued under s 28(1)(a) of the Privacy Act, are voluntary but represent the OAIC’s view on best practice with respect to undertaking data matching activities.

7 There are 13 Guidelines. Each Guideline includes summary text at the beginning to assist the reader.

8 The OAIC encourages agencies to agree to adopt and comply with the Guidelines. However, an agency that has so agreed would not be acting unlawfully if it did not comply, unless the acts or practices of the agency constitute a breach of the Privacy Act.

9 The APPs regulate the way in which agencies handle personal information. Under the Privacy Act, the OAIC has the power to conduct assessments, investigate complaints, or investigate on the Commissioner’s own initiative to determine whether agencies are complying with the APPs (see below at Role of the OAIC).

10The OAIC may take the Guidelines into account when assessing whether an agency has complied with the APPs. This document provides guidance on how to comply with the APPs when carrying out data matching activities, and guidance on other matters that may not be covered by the APPs. Accordingly, a breach of the Guidelines will not necessarily constitute a breach of the APPs.

11 The OAIC considers that the adoption of the Guidelines supports good privacy practice, reflects a commitment to the protection of individual privacy and promotes an Australian society in which privacy is respected.

12 The OAIC publishes information about government data matching activities in its annual reports, including a summary of program protocols received by the OAIC in each financial year.

13 These Guidelines are a resource for data matching activities; they are not specifically aimed at the related activities of data mining[11] and data analytics.[12] However, some data analytics and data mining may involve data matching activities covered by these Guidelines. The OAIC is aware that a number of agencies have implemented analytics and/or mining programs, and that these practices are likely to increase. All Australian Government agency data analytics or data mining programs which involve the use of personal information must comply with the APPs.

History of the Guide

14 In 1990, the Privacy Commissioner issued a consultation draft of The Use of Data Matching in Commonwealth Administration — Guidelines (former guidelines) for public comment.

15In 1992, the Privacy Commissioner released the former guidelines for adoption by agencies.

16 The former guidelines were subsequently revised and re-issued in November 1995, and again in February 1998.

17 This document implements changes required as a consequence of the evolving use of data matching by Australian Government as well as the implementation of the APPs which commenced on 12 March 2014.

Role of the OAIC

Investigations

18 The OAIC has the function of investigating possible breaches of the Privacy Act.[13]

19 If an individual considers that an agency or organisation covered by the Privacy Act has interfered with his or her privacy, they can complain to the OAIC. The OAIC may investigate and attempt to resolve the matter by conciliation between the parties.

20 The Commissioner may make determinations requiring the payment of compensation for damages or other remedies, such as the provision of access or making or issuing an apology.[14] The Commissioner can also accept enforceable undertakings and can seek civil penalties for serious and repeated breaches of the Privacy Act.

21The Commissioner has the power to initiate an investigation on their own initiative in appropriate circumstances, without first receiving a complaint.[15]

22 It is also open to the Commissioner to inform the Minister responsible for the Privacy Act of action that an agency needs to take in order to comply with the APPs.[16]

23 The Commissioner also has the power to conduct assessments of agencies covered by the Privacy Act, including whether an agency is maintaining and handling personal information it holds in accordance with the APPs.[17]

Data matching

24 The Commissioner has a number of specific functions under the Privacy Act with respect to data matching, including:

  1. ‘undertaking research into, and monitoring developments in, data processing and technology (including data matching and linkage) to ensure that any adverse effects of such developments on the privacy of individuals are minimised’[18]
  2. ‘examining a proposal for data matching or linkage that may involve an interference with the privacy of individuals, or which may otherwise have any adverse effects on the privacy of individuals’[19]

25 The Privacy Act also enables (and, in some cases, requires) the Commissioner to report to the Minister who administers the Privacy Act about the exercise of specific functions, including the functions set out above.[20]

Application of the Guidelines

What scale of data matching do the Guidelines apply to?

26 Guideline 1 provides that the Guidelines apply to data matching programs that include the comparison of two or more data sets, and at least two of the data sets each contain information about more than 5000 individuals.

Where the Guidelines do not apply

When should agencies report to the OAIC?

27 On request by the OAIC, each agency should provide the OAIC with a report or appropriate information on any data matching program that is not covered by these guidelines, for which they are the matching agency.

Privacy safeguards

28 If, in the course of participating in a data matching program not covered by the Guidelines, an agency becomes aware of information that it considers warrants administrative action being taken against an individual, the OAIC considers that it would be good privacy practice to inform the individual and offer them the opportunity to respond regarding the accuracy of the information the subject of the match (in accordance with Guideline 6 — Notify individuals of proposed administrative action, below).

Exemption from the Guidelines

29 Guideline 10 provides that in appropriate circumstances the Commissioner can grant an agency an exemption from specific requirements of the Guidelines. Exemptions are granted on the basis of public interest grounds set out by the matching agency.

Guideline 1: Application of the Guidelines

Guideline 1: Summary

1.1 Subject to the exceptions set out in Guideline 1.2, the Guidelines apply to a data matching program if:

  1. the data matching program includes the comparison of two or more data sets, and at least two of the data sets each contain information about more than 5000 individuals, and
  2. the data sets were collected for different purposes, and
  3. the purpose of the data matching program is:
    1. to select individuals for possible administrative action, or
    2. to add information from one database to another for purposes which include taking administrative action in relation to the individuals concerned, or
    3. to add information from one database to another with the intention of analysing the combined information to identify cases where further administrative action may be warranted, or
    4. to permanently combine the databases which provided the data sets being matched by the data matching program

1.2The Guidelines do not apply to a data matching program if:

  1. the objective of the data match is to verify personal information provided by the individual about their circumstances, status or relationships (as recorded by an agency) with another agency or organisation, and the result of the data match will not be used materially in making a determination around whether administrative action will be taken
  2. the objective is to co-locate records or data items previously held in separate locations, and the co-location does not result in any change to the purposes for which the records or data items are used or disclosed

Similar data matching programs to be treated as a single data matching program

1.3 If an agency runs several very similar data matching programs (for example, programs which have one data set and the algorithm in common, but vary as to the other data set), they should be treated as a single data matching program for the purpose of assessing whether the Guidelines apply. In particular, the other source data sets should be regarded as a single data set when deciding whether the data sets used in the data matching program contain records about more than 5000 individuals.

1.4 If an agency conducts a number of similar data matching programs that have the same objective and allow the drawing of similar inferences about the individuals identified, the agency should treat those programs as a single data matching program for the purpose of complying with this Guide.

Data matching programs involving more than one agency

1.5 Different agencies generally hold information for different purposes. Accordingly, where more than one agency participates in a data matching program, the Guidelines will likely apply to that program (see Guideline 1.1(b)).

Guideline 2: Deciding to carry out or participate in a data matching program

Guideline 2: summary

2.1 When deciding to carry out or participate in a data matching program, or to recommend that such a program should commence, an agency should take into account:

  1. the costs and benefits of the proposed data matching program
  2. whether there are any alternative measures to data matching that could achieve the same results as the proposed data matching program

Carrying out a data matching program

2.2 There is significant potential for data matching to pose a risk to an individual’s privacy. As such, the OAIC considers that it is best privacy practice to only carry out data matching where there is a clear business case, having regard to:

  1. the financial and non-financial costs and benefits (see Appendix C for guidance on how to assess the costs and benefits of a data matching program)
  2. whether the desired outcome could practicably be achieved by other means which pose less risk to an individuals’ privacy

Participating in a data matching program

2.3 The OAIC encourages agencies that are considering whether to participate in a data matching program (particularly new or large projects) to carry out a Privacy Impact Assessment of the project. The OAIC has published a Privacy Impact Assessment Guide[21] which provides guidance on, amongst other things, how to assess the privacy impacts of a project.

Guideline 3: Prepare a program protocol

Guideline 3: summary

3.1 Before commencing a data matching program, the primary user agency should:

  1. prepare a program protocol in accordance with Appendix A
  2. provide a copy of the program protocol to the OAIC
  3. make the program protocol publicly available

3.2 Each entity involved in a data matching program should ensure that its participation complies with the program protocol.

Purpose of the program protocol

3.3 The purpose of the program protocol is to inform the public about the existence and nature of the data matching program.

What should the program protocol contain?

3.4 The program protocol should provide the following:

  1. a description of the program, including:
    1. an overview of the program
    2. the objectives of the program
    3. the matching agency and source entities, and any agencies that will use the results of the program
    4. a description of the data to be provided and the methods used to ensure it is of sufficient quality for use in the program
    5. a brief description of the matching process, the output produced and the destination of the results of the program
    6. what action, administrative or otherwise, may be taken as a result of the program
    7. time limits applying to the conduct of the program
    8. what form of notice has been given, or is intended to be given, to individuals whose privacy is affected by the program
  2. an explanation of the reasons for deciding to conduct the program, including:
    1. the program’s relationship to the agencies’ lawful functions and activities
    2. the legal authority for the uses and disclosures of personal information involved in the program
    3. alternative measures to data matching that were considered, and the reasons why they were discounted
    4. information about any pilot testing of the program
    5. a statement of the costs and benefits of the program (see Appendix C for a description of what the statement should contain)

3.5 A suggested format, and more detailed guidance on the elements of a program protocol, are set out in Appendix A.

Program protocols for similar data matching programs treated as a single program

3.6 Where a number of similar data matching programs have been treated as a single data matching program under Guideline 1, the program protocol prepared to cover those data matching programs should deal with the matters set out in Appendix A and should also set out:

  1. each source entity and source data base or data set used
  2. how many different data matching programs are involved
  3. what classes of agency staff are responsible for conducting them

Publishing the program protocol

3.7 With respect to Guideline 3.1(c), the OAIC recommends that the primary user agency publish the program protocol for the data matching program on its website.

3.8 Agencies should also consider their obligations under the FOI Act, particularly in relation to the Information Publication Scheme (IPS). Under the IPS, agencies are required to publish information that falls within the categories specified in s 8(2) of the FOI Act. In particular, agencies should consider whether the program protocol for a data matching program falls within the scope of the agency’s ’operational information’,[22] and should therefore be published as part of the IPS.

3.9 Further guidance on the IPS is set out in Part 13 of the Guidelines Issued by the Australian Information Commissioner under s 93A of the Freedom of Information Act 1982.[23]

Changes to data matching programs

3.10 When a primary user agency wishes to change or amend an existing data matching program, the primary user agency should revise the program protocol to clearly indicate the amendments.

3.11 The primary user agency should notify the OAIC of the amendments and provide the OAIC with a copy of the revised program protocol.

3.12The amendments should be considered in the program evaluation (see Guideline 9 — Regularly evaluate data matching programs).

Guideline 4: Prepare a technical standards report

Guideline 4: summary

4.1 Before commencing a data matching program, the matching agency should prepare detailed technical standards to govern the conduct of the data matching program.

4.2 Where practical, the technical standards should be developed in consultation with source entities.

4.3 The matching agency should detail the technical standards in a technical standard report that includes the matters set out in Appendix B.

Purpose of the technical standards report

4.4 APP 11 requires that agencies take reasonable steps in the circumstances to protect information they hold from misuse, interference, loss and unauthorised use, modification or disclosure. Further, APP 10 and APP 13 require that agencies take reasonable steps in the circumstances to ensure the accuracy of the personal information they hold.

4.5Accordingly, the OAIC considers that it is best practice to have clearly expressed and detailed technical standards.

4.6 The Attorney-General’s Department has published the Improving the Integrity of Identity Data — Data Matching Better Practice Guidelines,[24] which provides guidance that may assist in establishing appropriate technical standards.

4.7 Where an agency conducts several data matching programs that are similar in operation, it may be reasonable for those programs to be governed by a single technical standards report.

What should the technical standards report contain?

4.8 The technical standards report should include the following:

  1. a description of data supplied by source entities
  2. the specification for each matching algorithm or project
  3. any risks inherent in the data matching program, and how those risks will be addressed
  4. controls to be employed to ensure the continued integrity of the data used in the data matching program, and of the data matching program as a whole
  5. security features included in the program to control and minimise access to personal information

4.9 Where an agency is unable to specify the matching algorithm used in a program, the agency should provide a general description of the matching algorithm and indicate the reasons why the matching algorithm cannot be specified.

4.10 More detailed guidance on the elements of a technical standards report is set out in Appendix B.

Comply with the technical standards report

4.11 The technical standards report should be prepared and held by the matching agency, and copies held by the source entities and user agencies where this is practicable.

4.12 Each entity participating in a data matching program (in any capacity) should ensure that its participation in the program is in accordance with the technical standards report.

4.13 The matching agency should provide a copy of the technical standards report to the OAIC on request. The technical standards report will assist in the proper assessment of an agency’s compliance with the APPs, in that it presents information in a manner that is capable of independent scrutiny. Accordingly, the technical standards report may be used as a basis for any review of the data matching activity that the OAIC may conduct (see Guideline 13 — Enable review by the OAIC).

New data matching programs

4.14 For new data matching programs, the technical standards report should be completed in draft form prior to the commencement of the program. It should be finalised not later than 30 days after the end of the first cycle, taking account of the initial experience of the operation of the program.

Changes to data matching program specifications

4.15When a matching agency wishes to amend the specifications of an existing data matching program, the matching agency should revise the technical standards report to clearly indicate the amendments.

Guideline 5: Notify the public

Guideline 5: summary

5.1Before an entity carries out or participates in a data matching program, the entity should take reasonable steps to ensure public notice of the proposed program is given.

Obligation to notify

5.2 APP 5 requires agencies that collect personal information about an individual to take reasonable steps in the circumstances, before the collection or as soon as practicable after, to notify the individual, or ensure they are aware, of certain matters including:

  1. the identity and contact details of the agency
  2. that the agency collects such personal information and the circumstances of that collection
  3. the purposes for which the agency collected the information
  4. the main consequences, if any, for the individual if all or some of the personal information is not collected by the agency
  5. to whom the agency usually discloses that kind of information
  6. that the agencies’ privacy policy contains information about access, correction and complaint handling
  7. details about any likely disclosures to overseas recipients and their locations

5.3 For many data matching activities covered by the Guidelines, it may not be a ‘reasonable’ step for an agency to provide each individual affected by the activity with a specific APP 5 notification. If this is the case, a public notice of the data match activity (as detailed below) is considered a ‘reasonable’ step for an agency to take to satisfy APP 5 obligations.

5.4 Agencies should also note that APP 3 applies different requirements to the collection of ‘sensitive’ information by an agency, compared to other types of personal information, including through a data match activity. Unless an exception applies, an APP entity may only collect sensitive information for a data match activity where the conditions of APP 3 are met and the individual concerned has consented to the collection.

Content of public notice

5.5 Public notification of a data matching program should, as a minimum:

  1. contain a brief description of the objectives of the data matching program
  2. list the matching agency and source entities involved in the data matching program
  3. contain a description of the data contained in the data sets involved in the data matching program
  4. list the categories of individuals about whom personal information is to be matched
  5. include the approximate number of individuals affected
  6. refer to the agency’s privacy policy

Forms of public notice

Primary user agency

5.6 The primary user agency for a proposed data matching program should cause a notice to be published in the Commonwealth Government Gazette (Gazette).

5.7 The Gazette notice should be published before the commencement of the data matching program.

5.8The primary user agency should forward copies of the Gazette notice to any individuals, agencies or organisations nominated by the OAIC (if any).

All participating agencies or source entities

5.9 Each agency or source entities that are organisations participating in a data matching program in any capacity should take reasonable steps to notify the general public and affected individuals about the data matching program.

5.10 In addition to a notice in the Gazette, agencies could notify the general public by, for example:

  1. including a notification on the agency’s website
  2. placing advertisements in print or online media publications

5.11 Agencies or source entities that are organisations should also take steps specifically aimed at informing individuals whose information is likely to be used in the program. This could be done by, for example:

  1. including information in a privacy policy about using and disclosing personal information for data matching purposes
  2. including information about the proposed data matching program in material given to individuals when they provide information that is likely to be used in the data matching program
  3. informing relevant individuals about the proposed data matching program directly (for example, by letter or email)
  4. by placing notices in relevant special-purpose publications or newsletters

5.12 Where a number of similar programs have been treated as a single program (see Guideline 1 – Application of the Guidelines), the Gazette notice and other publicity material should deal with the matters set out above (see Content of public notice above) and should also describe:

  1. the range of data sources used
  2. how many different data matching programs are involved
  3. what classes of agency staff are responsible for conducting those data matching programs

5.13 The Gazette notice and other publicity material should advise how the general public can obtain copies of the program protocol.

Privacy policy

5.14 APP 1.3 requires entities to have a privacy policy about how they manage personal information. APP 1.4 specifies that an entity’s privacy policy must contain information including the following:

  1. the kinds of personal information the agency collects and holds
  2. how the entity collects and holds that information
  3. the purpose for which the entity collects, holds, uses and discloses that information
  4. details of how individuals can get access to information about themselves and seek correction of such information

5.15 APP 1.5 requires entities to take reasonable steps in the circumstances to make its privacy policy available free of charge and in an appropriate form. This will usually involve agencies making their privacy policy available on their websites.

5.16 Where an entity collects, uses and discloses personal information for data matching purposes:

  • it could include a description of personal information it holds in connection with a data matching program in its privacy policy
  • a source entity could include a note in its privacy policy that the information it holds is disclosed to the matching agency in connection with a data matching program

Guideline 6: Notify individuals of proposed administrative action

Guideline 6: summary

6.1 Before taking administrative action against an individual in response to a match, including checking information with third parties, agencies should notify the individual of the match and the proposed administrative action, and give the individual the opportunity to respond.

Notify individuals

6.2 APP 10 requires that agencies ensure the accuracy of the personal information they hold before using that information. APP 13 requires agencies to take steps to amend records that contain personal information to ensure that personal information is accurate.

6.3 In relation to a match produced as the result of a data matching program, methods to ensure the accuracy of the match could include checking the data with third parties, against source data, or with the individual who is the subject of the match.

6.4 A user agency should only take administrative action in response to a match after giving the individual concerned:

  1. reasonable notice of the relevant matters, including:
    • the match
    • the initial conclusions the agency has drawn based on the match
    • the administrative action that the agency proposes to take in response to the match, and
  2. a reasonable period (for example 14 days) in which to respond to that information

6.5 If there is a dispute as to the accuracy of the data, but the agency considers that administrative action is still warranted, it should inform the individual of their right to lodge a complaint with the OAIC regarding the accuracy of their personal information.

6.6 If an agency plans to take administrative action in response to a match without notifying the individual concerned of the match (for example, as part of a data matching program that does not comply with the Guidelines on public interest grounds; see Guideline 10 — Seeking exemptions from Guideline requirements), the agency should take reasonable steps to ensure the accuracy of the information before taking action.

6.7 The decision to take administrative action should ideally involve consideration of the circumstances of each case. However, where a program is generating large numbers of matches, it may be reasonable to apply some rule-based selection criteria, to achieve further filtering or selection of cases independently of the matching program itself. The filtering criteria should be described in the technical standards report (see Guideline 4 — Prepare a technical standards report).

6.8 Unless required or authorised by law, an agency should not take administrative action that interferes with the individual’s opportunity to exercise any rights of appeal or review.

Guideline 7: Destroy information that is no longer required

Guideline 7: summary

7.1Destroy or de-identify personal information that is no longer required.

Destroy information that is no longer required

7.2 APP 11.2 requires an agency to take reasonable steps in the circumstances to destroy or de-identify personal information it holds where it no longer needs the information for any purpose for which the information may be used or disclosed, the information is not held in a Commonwealth record and the agency is not required by law or court/tribunal order to retain the information.

7.3In order to comply with this Guideline, generally destruction of records collected for the purpose of data matching should be conducted in accordance with the National Archives of Australia’s General Disposal Authority 24 – Records Relating to Data Matching Exercises (the Disposal Authority).[25]

7.4 That document provides that:

  1. where personal information is obtained for use in a ’data matching exercise’ (ie a data matching cycle as part of a data matching program), and
  2. the ’data matching process’ (ie the data matching cycle involving the information) does not lead to a match

then the personal information that is no longer required should be destroyed by the matching agency as soon as practicable after the completion of the data matching cycle, and at least within 90 days of the completion of the data matching cycle.

7.5For the purposes of this Guideline, where a match occurs in the data matching cycle, a decision as to further action in relation to each individual should be taken by an agency within 90 days of the data matching cycle.

7.6 Further, where personal information is obtained for use in a ‘data matching exercise’ and the ‘data matching process’ leads to a match, then if, during that time or any later time, a decision is made not to take further action, the information should be destroyed as soon as practicable after, but at least within 90 days of, the decision not to take further action. This applies to each agency using the results of the data matching program.

7.7 The Disposal Authority provides that the Commissioner may approve an extension of time for the destruction of such information. Agencies that seek such approval must make a written request to the OAIC. The request should be made in accordance with Guideline 10 — Seeking exemptions from Guideline requirements.

Guideline 8: Do not create new registers, data sets, or databases

Guideline 8: summary

8.1 An agency involved in a data matching program, in any capacity, should not create any new separate permanent register, data set, or database using data sets or information contained in data sets collected as part of the data matching program.

Do not create new registers

8.2 APP 6 provides that agencies must only use personal information for the particular purpose for which it was collected, or for a secondary purpose if an exception applies.

8.3 Privacy concerns relating to data matching include the possibility that agencies will:

  1. use personal information collected for the purpose of a specific data matching program for an unrelated secondary purpose
  2. retain personal information indefinitely in case it becomes useful in future

8.4 Unless a secondary use or disclosure is authorised by an exception listed in APP 6, personal information collected for the purpose of data matching should be destroyed when no longer required (see Guideline 7 — Destroy information that is no longer required).

8.5 Compliance with Guideline 8 would not preclude:

  1. the maintenance of a register of individuals in respect of whom further investigations are warranted under the terms of the program protocol, following a decision to take administrative action involving those individuals
  2. the maintenance of a special register solely for the purpose of excluding individuals from being selected for investigation in successive data matching cycles of the same data matching program
  3. the maintenance of such records or data sets as is reasonably necessary to achieve the objectives of the program

Guideline 9: Regularly evaluate data matching programs

Guideline 9: summary

9.1The primary user agency should evaluate the conduct and outcomes of data matching programs no later than three years after the commencement of operation of the data matching program, and at least every three years after that while the program continues.

Regular evaluation

9.2 Prior to participating in a data matching program, agencies should ensure that their decision to participate is based on a sound business case, and is in the public interest having regard to the potential for data matching to be privacy invasive (see Guideline 2 — Deciding to carry out or participate in a data matching program and Guideline 3 — Prepare a program protocol).

9.3 However, where data matching programs are conducted over the course of several years, it is important for agencies to periodically confirm that the reasons for participating in the data matching program are still valid – for example, that the program is achieving its objectives and has not deviated from the privacy and data quality safeguards specified in the program protocol and technical standard report.

9.4 Accordingly, no less than every three years, the primary user agency should undertake an evaluation of the data matching program in accordance with its original objectives.

9.5 Where the primary user agency is not the matching agency, the primary user agency should consult with the matching agency regarding the evaluation.

9.6 The evaluation should include (as a minimum):

  1. consideration of whether the data matching program has achieved its objectives
  2. consideration of whether the program has complied with the program protocols and technical standards report
  3. consideration of whether the privacy and data quality safeguards incorporated into the data matching program have been effective
  4. a revised statement of the costs and benefits of the program (see Appendix C: Statement of costs and benefits for data matching programs)
  5. a determination as to whether the reasons for conducting the data matching program are still valid, and details of any changes or amendments to the data matching program during the evaluation period, or as a consequence of the evaluation

9.7 The primary user agency should:

  1. document the conclusions of the evaluation in a report
  2. make a report of the evaluation publicly available (by, for example, posting a copy on the agency’s website)
  3. provide a copy of the report to the OAIC

9.8 Note: Where the data matching program has not been publicly notified (for example, where the data matching program relates to confidential information sourced from law enforcement or national security agencies) it may not be appropriate to make a copy of the evaluation report publicly available.

Guideline 10: Seeking exemptions from Guideline requirements

Guideline 10: summary

10.1 Where the head of an agency considers that it would be appropriate (having regard to the public interest) to conduct a data matching program to which the Guidelines apply in a way that would be inconsistent with one or more of the Guidelines, they should:

  1. advise the Commissioner in writing of the details of the proposed data matching program
  2. in that advice, specify how the proposed data matching program would be inconsistent with the Guidelines, and
  3. explain the public interest grounds that justify the inconsistency

Explaining the public interest grounds

10.2 In explaining the public interest grounds, the head of the agency should address matters such as the following:

  1. the effect that not abiding by the Guidelines would have on individual privacy
  2. the seriousness of the administrative or enforcement action that may flow from a match obtained through the data matching program
  3. the effect that not abiding by the Guidelines would have on the fairness of the data matching program — including its effect on the ability of individuals to determine the basis of decisions that affect them, and their ability to dispute those decisions
  4. the effect that not abiding by the Guidelines would have on the transparency and accountability of agency and government operations
  5. the effect that not abiding by the Guidelines would have on compliance of the proposed data matching program with the APPs
  6. the effect that complying with the Guidelines would have on the effectiveness of the proposed data matching program
  7. whether complying fully with the Guidelines could jeopardise or endanger the life or physical safety of information providers or could compromise the source of information provided in confidence
  8. the effect that complying fully with the Guidelines would have on public revenue – including tax revenue, personal benefit payments, debts to the Commonwealth and fraud against the Commonwealth
  9. whether complying fully with the Guidelines would involve the release of a document that would be an exempt document under the FOI Act
  10. any legal authority for, or any legal obligation that requires, the conduct of the proposed data matching program in a way that is inconsistent with the Guidelines

OAIC response to advice

10.3 The Commissioner may respond to the agency head’s advice, setting out their view as to whether it would be appropriate, from a privacy protection perspective, for the Guidelines not to be followed, and the reasons for taking this view.

10.4 If the Commissioner takes the view that it would be inappropriate, from a privacy protection perspective, for the Guidelines not to be followed, the Commissioner may suggest changes to the proposed data matching program which would, in the Commissioner’s view, achieve an adequate standard of privacy protection.

10.5 The Commissioner cannot exempt an agency from the requirements of the APPs in the absence of a relevant Public Interest Determination.[26]

Publication of advice

10.6 It is the normal practice of the OAIC to make an advice provided in accordance with Guideline 10 publicly available. However, the OAIC will keep such advice confidential if, in the advice, the agency head:

  1. requests that the advice remain confidential
  2. provides reasons for that request that the Commissioner considers to be sufficient

10.7 Freedom of Information requests for advice provided on a confidential basis will be considered in accordance with the FOI Act.

Guideline 11: Data matching with entities other than agencies

Guideline 11: summary

11.1 Where an agency proposes to carry out a data matching program that involves an entity that is not an agency (such as a State or Territory government body, or a private sector organisation), the agency should, where practicable, require that the entity adopt these Guidelines in respect of the data matching program.

Entities other than agencies

11.2 Under s 95B of the Privacy Act, agencies seeking to carry out a data matching activity with an organisation (under contract) must take contractual measures to ensure that the organisation carries out the contract work in a manner consistent with the APPs.

11.3 A ‘contractual measure’ in this context might include, where practicable, requiring non-agency participants in a data matching program to adopt these Guidelines.

11.4 For example, where an agency enters into a contractual agreement to carry out data matching with an organisation (for example, where an organisation has agreed to be a source entity), the agency could seek that the contract includes a condition requiring the parties to the contract to adopt these Guidelines.

11.5 Where an agency uses statutory powers to compel the participation of an organisation in a data matching program (for example, to compel the organisation to supply a data set), it may be impracticable to require the organisation to adopt these Guidelines.

Guideline 12: Data matching with exempt agencies

Guideline 12: summary

12.1 Where an agency participates in a data matching program with an agency that is exempt under s 7 of the Privacy Act, the first agency should take steps to ensure its own compliance with the APPs and the Guidelines (if applicable).

Exempt agencies

12.2 Section 7 of the Privacy Act provides that the acts and practices of specified entities, including certain agencies, are not covered by the Privacy Act (for example, the Australian Crime Commission or the Australian Security Intelligence Organisation).

12.3 An agency that is covered by the Privacy Act (non-exempt agency) may engage in data matching with an agency which is exempt under s 7 of the Privacy Act (exempt agency). For example, the non-exempt agency may be a source entity for a matching agency that is an exempt agency. However, the fact that a data matching program includes an exempt agency does not relieve a participating non-exempt agency of its obligations under the APPs or the Guidelines, (if applicable).

12.4As detailed at paragraph 5 in the ‘Background’ section, an agency will not be taken to have acted inconsistently with the Guidelines by virtue of an action that the agency is required or specifically authorised by law to take.

12.5 Before participating in a data matching program with an exempt agency, a non-exempt agency should ensure that arrangements are in place which allow it to comply with the APPs and the Guidelines (if applicable).

12.6 As a matter of best practice, exempt agencies could adopt and comply with these Guidelines, so far as is consistent with the agency’s exempt status.

Guideline 13: Enable review by the OAIC

Guideline 13: summary

13.1 Agencies should enable the OAIC to review their data matching activities and their procedures.

OAIC review

13.2 The review process carried out by the OAIC may include:

  1. assessing whether the data matching program is being conducted in accordance with the procedures set out in the program protocol
  2. reviewing the effectiveness of the controls and procedures set out in the technical standards report
  3. assessing the outcomes of the data matching program from a privacy perspective
  4. considering any complaints and difficulties that have arisen in connection with the data matching program

13.3 On request from the OAIC, agencies should report to the OAIC on any relevant matter, including:

  1. actual costs and benefits flowing from the data matching program
  2. any non-financial but quantifiable factors that are considered relevant
  3. any difficulties in the operation of the program and how these have been overcome
  4. the extent to which internal audits or other forms of assessment have been undertaken by the participating agencies or organisations, and their outcome
  5. the number of matches produced, the number of matches investigated, the number of cases not proceeded with after contacting the affected individual, and the number of cases in which action proceeded despite a challenge as to accuracy of the data

13.4 It is the usual practice of the OAIC to include in its annual report general information about:

  1. the number, extent and nature of data matching programs
  2. the extent of public notification of programs and of consultation
  3. the extent of confidential notification of programs to the OAIC
  4. the nature of the public interest reasons advanced for not engaging in public notification of programs
  5. the operational experience and effectiveness of programs

Appendix A: Content of data matching program protocols

The purpose of the program protocol is to inform the public about the existence and nature of the data matching program.

Accordingly, the program protocol should be written in plain English, and give an accurate picture of how the data matching program works.

The program protocol should provide the following:

  • a description of the data matching program, including:
    • an overview of the data matching program
    • the objectives of the data matching program
    • the matching agency and source entities, and any agencies that will use the results of the data matching program
    • a description of the data to be provided, and the methods used to ensure it is of sufficient quality for use in the data matching program
    • a brief description of the matching process, the output produced and the destination of the results of the data matching program
    • what action, administrative or otherwise, may be taken as a result of the data matching program
    • time limits applying to the conduct of the data matching program
    • what form of notice has been given, or is intended to be given, to individuals whose privacy is affected by the data matching program
    • if the agency is writing to the OAIC to depart from the Guidelines on public interest grounds under Guideline 10; the correspondence to the OAIC should also accompany the protocol
  • an explanation of the reasons for deciding to conduct the data matching program, including:
    • the data matching program’s relationship to the agencies’ lawful functions and activities
    • the legal authority for the uses and disclosures of personal information involved in the data matching program
    • alternative measures to data matching that were considered, and the reasons why they were rejected
    • information about any pilot testing of the data matching program
    • a statement of the costs and benefits of the data matching program (see Appendix C: Statement of costs and benefits for data matching programs Appendix B: Technical standards report for a description of what the statement should contain)

A suggested format, and some guidance on the elements of a program protocol, is set out below.

Description of the data matching program

The description of the data matching program needs to cover the following matters. Suggested section titles are in parentheses.

An overview of the data matching program (Overview)

A short, simply expressed statement of what the data matching program does and why. Two to three hundred words should be sufficient.

The objectives of the data matching program (Objectives)

A basic statement of what the data matching program is trying to achieve.

The rest of the protocol will outline how the data matching program will try to achieve its objectives, so this statement does not need to be lengthy.

The matching agency and source entities, and any agencies which will use the results of the data matching program (Agencies involved)

This should include:

  • which agency is conducting the data matching program
  • where the matching agency is not the same as the primary user agency, the protocol should clearly establish the functions and roles of each participating agency with respect to the data matching program
  • which source entities are providing data that will be used in the data matching program – this should cover all sources of data, including non-Commonwealth and non-government sources, and
  • all entities that have access to the results of the data matching program

A description of the data to be provided and the methods used to ensure it is of sufficient quality for use in the data matching program (Data issues)

For each data or information source involved in the data matching program, briefly describe:

  • the kind of files transferred to the matching agency
  • the type of information contained in the file
  • the approximate number of records on each file, and
  • what measures have been taken to ensure the quality, integrity and security of the data

A brief description of the matching process, the output produced and the destination of the results of the data matching program (The matching process)

Describe:

  • which fields are matched (eg agency identifier, name and date of birth)
  • what criteria are used to identify a ‘match’ (eg individuals on both files, individuals on one but not the other), and
  • the fields included in each output file

The specific technical details of the matching process will be set out in the technical standards report, so the description here can be made in relatively broad terms.

What action, administrative or otherwise, may be taken as a result of the data matching program (Action resulting from the program)

This should cover all agencies that use the results of the matching. If an agency may take one of a range of actions, depending on the facts of a particular case, each should be outlined. Copies of template letters to people that are proposed to be, or will be, the subject of administrative action should be attached to the program protocol as an appendix.

Time limits applying to the conduct of the data matching program (Time limits)

This should cover:

  • how long data obtained for use in the data matching program will be kept, including both input data provided by source agencies and the output data from the matching
  • retention periods and disposal arrangements for all data
  • how frequently the program will be run, and if the data matching program will be run at infrequent intervals, how it is decided that a run is appropriate at a particular time, and
  • when it is planned to terminate or review the data matching program, including agencies’ internal review mechanisms as well as external mechanisms, such as legislative sunset clauses

What form of notice has been given, or is intended to be given, to individuals whose privacy is affected by the data matching program (Public notice of the program)

The text of all public notices, including APP 5 notices, Gazette notices and media releases, should be attached to the protocol.

If the agency is writing to the OAIC to depart from the Guidelines on public interest grounds under Guideline 10 (Departure from Guidelines)

Any departure from the Guidelines on public interests grounds should be explained, and the specific exemption sought. Any correspondence on the matter between the entity and the OAIC should be attached to the protocol.

Reasons for deciding to conduct the data matching program

The reasons for deciding to conduct the data matching program should cover the following matters.

The data matching program’s relationship to the agencies’ lawful functions and activities (Relationship to lawful functions)

This should clearly specify the link between the objectives of the data matching program and each agency’s lawful functions and activities.

The legal authority for the uses and disclosure of personal information involved in the data matching program (Legal authority)

The reasons should include the justification of the use and disclosure of personal information in terms of the Privacy Act and any other relevant legislation.

To be lawful, any use or disclosure of personal information by an agency for a purpose other than the particular purpose for which the information was collected must fall within one of the exceptions listed in APP 6.

The protocol should specify which exceptions in APP 6 apply, and why.

If there are other legislative requirements regarding the use and disclosure of information involved in the data matching program, such as secrecy provisions, the protocol should explain how the uses and disclosures are authorised in terms of the relevant provisions.

Alternative measures to data matching that were considered, and the reasons why they were rejected (Alternative methods

If it is considered that there are no practicable alternatives to data matching, the protocol should include a brief explanation of why this is the case.

Information about any pilot testing of the program (Pilot programs)

Where a pilot program has been conducted, the following information is likely to be relevant and should be included:

  • the number of records involved in the pilot program
  • the number of matches that resulted
  • an estimate or report of the benefits of the pilot program (if the matches were acted upon, it may be possible to give a detailed account of the benefits; if the matches were not acted upon, an estimate of the benefits that would have resulted should be given), and
  • information about any problems or difficulties with the matching program that was obtained from the pilot program

If the protocol relates to a new data matching program (rather than one already operating) and no pilot project has been conducted or is planned, the protocol should indicate why a pilot program is not considered to be necessary.

A statement of the costs and benefits of the data matching program (Costs and benefits)

See Appendix C: Statement of costs and benefits for data matching programs regarding the preparation of a statement of costs and benefits.

Appendix B: Technical standards report

Purpose

The purpose of the technical standards report is:

  • to ensure that data matching is conducted on the basis of pre-defined standards, including data-quality and security controls
  • to form a basis for any review of the actual data matching activity that the OAIC may conduct

Contents

The technical standards report should include:

  1. a description of data supplied by source entities, referring in particular to:
    • key terms and definitions
    • the scope and completeness of the data to be collected
    • the relevance of the data to be collected
    • the timing of the collection
  2. the specification for each matching algorithm or project, including such things as:
    • data items used in the match, particularly the use of any Government identifiers
    • the rules for recognising a match
    • the destination of the results of the data matching program
  3. any risks inherent in the data matching program, and how those risks will be addressed
  4. controls to be employed to ensure the continued integrity of the data used in the data matching program, and of the data matching program as a whole
  5. security features included in the program to control and minimise access to personal information

a. Description of data

As part of the technical standards report, the matching agency should, in consultation with the source entities where that is practicable, compile a ‘data dictionary’ for all data that is supplied as part of the data matching program, that includes:

  • a description of each file used by the matching agency, that outlines its source, destination and, for an intermediate file, its use
  • for each data item:
    • its name, description, the validation or edits applied to it
    • whether or not it has been standardised
    • the level of precision of the field, for example, YY or YY/MM/DD, annual income, amount in thousands

b. Matching techniques

The technical standards report should clearly document the following information about the data matching techniques to be used in the data matching program:

  • the matching algorithm used — for example, first six characters of family name and value of given name, together with date of birth; phonetic equivalent of family name and birth year
  • rules for recognising matches
  • the destination of matching results
  • the sampling techniques used to verify the validity/accuracy of matches
  • the techniques adopted to overcome identifiable problems with the quality of data and to standardise data items that have been compared but have different meaning (for example, ‘annual income’ and ‘financial year income’)

c. Risks

The technical standards report should identify any risks posed by the data matching program including, but not limited to, risks to the privacy of individuals, reputational risks, and risks relating to incorrect matches.

The technical standards report should clearly set out how those risks will be mitigated. Some risks may be mitigated through data quality and security controls (see below at (d) and (e)).

d. Data quality controls and audit

The technical standards report should clearly document:

  • any relevant measures taken to ensure data quality (for example, the timing of any extract files that may be taken for the data matching program)
  • any audit processes to which the data used in the data matching program has been, or is regularly, subjected

e. Security and confidentiality

The technical standards report should clearly document the precautions proposed to be taken at all stages of a data matching program to ensure that personal information used in and arising from a data matching program:

  • is not subject to accidental or intentional modification
  • is not accessed by staff within the agency except where such access is necessary for the conduct of that data matching program or resulting action
  • is not disclosed otherwise than as is intended by the program protocol

The technical standards report should make specific reference to access controls such as password security, encryption, and audit trails, including logging of access.

The technical standards report should also contain a list of all computer programs developed by the matching agency in relation to the data matching activity, together with a description of the functions of the programs.

Appendix C: Statement of costs and benefits for data matching programs

Introduction

Guideline 2 recommends that, prior to deciding to participate in or conduct a data matching program, agencies should take into account the estimated costs and benefits of conducting a data matching program. This is to ensure that the data matching program is an efficient use of Commonwealth resources, and to provide a level of transparency and accountability.

As a matter of best practice, the OAIC recommends that agencies prepare a statement of costs and benefits as outlined in this Appendix:

  • when starting a new data matching program (as part of the program protocol), and
  • when evaluating data matching programs

While it is desirable for the statement to be as comprehensive and rigorous as possible, it is not intended to be a formal cost-benefit analysis. In some cases the information required for such an analysis will not be available. In others, the type of net benefit bottom line that a formal cost-benefit analysis aims to produce will not be the most meaningful way of presenting the impact of the data matching program.

The appropriate degree of detail will vary depending on the nature of the data matching program. For example, a less detailed statement might be appropriate for a data matching program whose function is to carry out a task that would otherwise have to be performed manually. Similarly, a detailed statement may not be required where an agency routinely conducts similar data matching programs, and has previously given consideration to the costs and benefits.

The costs and benefits typically associated with data matching programs are categorised below, together with suggestions on how to present information in each category. The categories can be used as a checklist by agencies. However, it is important to note that not all of the categories of costs and benefits below will apply to every data matching program.

Purpose of estimating costs and benefits

The purpose of including a statement of costs and benefits in program protocols and program evaluations is:

  • to explain the agency’s reasons for determining that the data matching program is in the public interest
  • to help identify areas of potential risk (such as cost, legal liability or public sensitivity)
  • to provide a basis for evaluating the performance of the data matching program

A statement that:

  • provides aggregate figures for costs and benefits, and
  • explains how those figures were calculated

will be more useful and informative to readers, and will provide a better basis for comparison with actual performance, than a statement that only provides aggregate figures.

Where a statement provides estimated costs and benefits, it is critical that the statement clearly specifies the method by which the estimates were reached, and any assumptions on which they were based.

Methods of presenting cost/benefit information

Perspective

The OAIC suggests that statements of costs and benefits be presented from the perspective of the Commonwealth, rather than from the perspective of the individual agency or the wider community. This means that statements should present information about all significant costs and benefits to the Commonwealth, including costs and benefits experienced by more than one agency.

If a data matching program will have major costs or benefits for other parts of the community (sometimes called ‘externalities’), this should be noted in the statement.

Sources of information

Key sources of data on costs and benefits will be:

  • for data matching programs that are already running, data obtained from the actual operation of the program. This should include (in addition to more detailed information on costs and benefits) basic data on:
    • the total number of matches
    • the number of cases in which matches led to further investigation
    • the outcomes from investigation of cases
  • for new data matching programs, any pilot program or other preliminary assessment of the data matching program. If estimates of costs and benefits are based on results from pilot program, those results should be included

While international comparisons may be useful in limited circumstances, the grounds of comparison are rarely firm. Countries inevitably differ in numerous ways – cultural, economic, legal and so forth – that make parallels difficult to sustain. Such comparisons should be used with caution, if at all, and should not play a pivotal role in arriving at estimates of costs and benefits.

Methods of presenting information

The best way of presenting the information will depend on the nature of the data matching program and the information available. However, at a minimum, the statement should compare the outcomes from the data matching program and the outcomes that would arise in the absence of the data matching program.

Agencies could consider the following formats for statements of costs and benefits:

  • Compare the costs and benefits of the data matching program with the most likely alternative use of the resources required — that is, what those resources would be used for if the program did not go ahead. The benefits of the alternative use of resources are, effectively, the cost of carrying out the program (as they represent the opportunity cost of devoting those resources to the program). For example, if the resources used in the program have been diverted from a random audit program, then that would be the appropriate basis for comparison.
  • Compare the costs and benefits of conducting the data matching program against the costs and benefits of achieving the same result by some alternative method (for example, a manual process). This may be most appropriate where:
    • one of the main benefits of the data matching program is administrative savings from the more efficient performance of a task that would otherwise have to be carried out by other means, or
    • the benefits of the data matching program are quantifiable, but are not financial (for example, detection and prosecution of people breaking a law).
  • If the alternative uses of the resources required for the data matching program, or the costs of achieving the same outcomes by alternative means, cannot be ascertained, estimate the actual costs of the resources to be used in the program, and the expected benefits from the program, without making a comparison with an alternative scenario. This option should only be used where it is impossible to apply either of the other approaches; it gives much less useful information about the data matching program and makes it much more difficult to judge whether it constitutes an efficient application of resources. If this approach is taken, the statement should include the reasons why neither of the other approaches could be taken.

Calculating net present value

Formal techniques for cost/benefit analysis often include provision for ‘discounting’ of costs and benefits that occur later, to arrive at a net present value for a data matching program.

This type of calculation is only necessary for data matching programs that have a high establishment cost (for example, if they involve significant capital expenditure) and long-term benefits.

If costs and benefits are discounted, the undiscounted values should still be presented in the statement, and the discount rate explicitly stated.

Information on calculating net present values can be found on page 18 of the Department of Finance and Deregulation publication Introduction to Cost-Benefit Analysis and Alternative Evaluation Methodologies (January 2006).[27]

Further information on cost/benefit analysis

The Department of Finance and Deregulation has produced the following publications relevant to cost/benefit assessment:

  • Guidelines for Costing of Government Activities (1994)[28]
  • Introduction to Cost-Benefit Analysis and Alternative Evaluation Methodologies (January 2006)
  • Handbook of Cost-Benefit Analysis (January 2006)[29]

Estimating costs

The costs of data matching programs can be broadly divided into:

  1. establishment costs, comprising:
    1. staff costs involved in setting up the data matching program – for example, staff time to develop appropriate systems to process and handle the results
    2. capital costs
    3. other costs, such as publicity costs and IT services
  2. running costs, including:
    1. costs associated with conducting runs of the data matching program – for example, the cost of maintaining the system
    2. costs associated with taking action in response to matches, such as administrative costs associated with corresponding with affected individuals or amending benefits, legal costs or the costs of appeals processes

For many data matching programs, the costs associated with some of these categories may be negligible (for example, many data matching programs do not involve capital expenditure) and can be ignored.

a. Establishment costs

Staff costs

The OAIC suggests the following approach to estimating the cost of staff time involved in developing a new data matching project:

Step 1: Estimate the amount of time (in person weeks, months or years) that will be spent by all staff on initial development of the project.

Step 2: Estimate the average salary of project staff (per week, month or year), and multiply it by the estimated staff time needed for project development.

Step 3: Multiply the result of Step 2 by a factor to allow for labour on-costs and overheads. The Department of Finance and Deregulation recommends multiplying the basic salary cost by a factor of 2.54 to get the total staff costs (see Chapter 5 of the Department of Finance Guidelines for the Costing of Government Activities).

The staff costs must include time spent on the project by all staff, including administrative, corporate services, IT and support staff, as well as staff dedicated to the project. It is not acceptable to minimise the apparent costs of a data matching program by trying to shift costs associated with the program to other areas of the agency.

The statement of costs and benefits should indicate the amount of staff time estimated for project development, the total staff cost of setting up the project and the method of calculation.

Capital costs

Most data matching programs do not require capital expenditure. If capital outlays are required for a program (for example, if computer facilities are to be expanded to cater for the project) they should be stated (see page 13 of the Department of Finance and Deregulation Guidelines for Costing of Government Activities for more information on assessing capital costs).

Other costs

It is not necessary to include other costs, such as publicity costs and use of computer facilities unless they are significant in magnitude. As a general rule, it will not be necessary to quantify such costs if they represent only a few per cent of overall establishment costs.

Agencies should ensure that all other costs, including other costs incurred by IT, corporate services or special projects areas are considered in this category.

b. Running costs

Cost of conducting matching

The cost of conducting the data matching program would include:

  • staff time (of both IT staff and administrative staff with continuing responsibility for managing the program)
  • IT services

If these costs are negligible they may be excluded from the statement of costs; a statement that the costs are negligible and have been excluded should be included in the protocol.

Costs associated with responding to matches

The costs associated with taking action in response to matches obtained through a data matching program will also tend to be predominantly comprised of staff costs.

The method used to estimate those costs will depend on the way the data matching program results are used.

If the response to matches is carried out by dedicated staff, the cost of this activity can be readily calculated.

If staff who carry out reviews based on matches also have other functions, the time required for those reviews could be calculated either by:

  • estimating the proportion of time that the review staff spend on this activity
  • estimating the time required for an average review, and multiplying it by the number of reviews undertaken

The latter approach would be most suitable in situations where the task of responding to matches is decentralised, and data on how review staff allocate their time is not available.

Estimating benefits

The benefits of data matching programs can be broadly divided into:

  1. direct financial benefits, including
    • recovery of incorrect payments
    • prevention of incorrect payments
    • increased revenue collection
  2. indirect financial benefits, such as:
  • administrative savings
  • benefits of voluntary compliance (deterrence effects), and
  • non-financial benefits

a. Direct financial benefits

Direct financial benefits will mainly fall into the following three categories.

Recovery of incorrect payments

The total amount of overpayments identified as a result of the program should be reduced to recognise:

  • that not all overpayments identified will be recovered, because some amounts are too small to warrant recovery action, and in some cases because recovery action will be unsuccessful
  • the cost of recovery action (unless this is explicitly included under the costs of the program)

For example, an agency may estimate that 70 per cent of identified overpayments represent actual savings; the remaining 30 per cent of overpayments will be unable to be recovered, or the recovery costs would exceed the amount to be recovered.

The statement of benefits should include both the total amount of overpayments identified and the method of calculating how much of those overpayments represent actual savings.

If full figures for recovered amounts and the costs of recovery are available (either from a pilot project in the case of a new program or from experience with a program being evaluated), these could be used rather than adopting the approach outlined above.

Avoidance of incorrect payments

This may occur, for example, where a data matching program identifies that someone currently receiving a Government payment is not entitled to it, or is not entitled to payment at the current rate, and this leads to termination or reduction of the payment.

Especially in the case of continuing payments (for example, welfare benefits) it will often not be possible to conclusively determine how much would have been incorrectly paid had the payment not been terminated or reduced.

If a general assumption is made (for example, that the incorrect amount would have continued to be paid for a standard period), the statement should clearly specify the assumption, and the reasons for its adoption. For example, an agency might assume that an incorrect payment would have continued to be paid for half the average period for which payments of that type are made, based on past experience regarding the average time required to identify incorrect payments.

In making such an assumption, agencies should account for the possibility that other review methods (if relevant or applicable) could have identified the incorrect payment had the data matching program not done so.

Increasing the revenue collected by an agency

If a program identifies cases where additional revenue is owed to an agency, the estimate of the benefit derived should either:

  • allow for the possibility that all revenue owed will not be collected, and the cost of collection (unless the cost of collection is included as a cost of the program)
  • be based on the actual amounts collected as a result of the data matching program and the costs of collection

b. Indirect financial benefits

Administrative savings

Savings of this sort are likely to be most important where data matching allows an activity that would have to be carried out in any case, to be performed more efficiently.

One approach to estimating these savings would be to estimate the comparative cost of carrying out the activity with and without data matching (see the section above titled ’Methods of presenting costs/benefit information’).

Voluntary compliance

Agencies may think that public knowledge that a data matching program is operating leads to increased compliance with the law, thus reducing regulatory costs. This kind of benefit is obviously difficult to quantify. If agencies believe that benefits of this sort are likely to be achieved, they should include them in the statement of benefits, along with the reasons for holding this view and any information that indicates the likely magnitude of benefits from this source. This would require estimates of current error rates or fraud rates together with an assessment of the program’s anticipated impact on them.

c. Non-financial benefits

Many data matching programs have benefits that cannot readily be expressed in financial terms. For example, data matching is used to facilitate visa compliance, to detect criminal offences and to build up intelligence holdings of law enforcement agencies.

Some types of benefits probably cannot be quantified at all, but should still be described. For example, if a benefit of data matching is improved services to clients or improved data quality, the statement of benefits could describe the effect of the data matching program in these regards.

Where possible, it is helpful to quantify non-financial benefits. For example, if a program is aimed at facilitating visa compliance, it may be useful to state how many visa overstayers have been, or are expected to be, located by means of the program. This would help to illustrate the reasons why a data matching program is considered worthwhile, and provide a basis for comparing actual performance against initial estimates.

If a data matching program does not have a readily quantifiable outcome of this sort, other measures of performance can be found. For example, if the function of a program is to add significant items of information to an intelligence database, it may be relevant to estimate how many items of information the program will identify. If matches contribute to an outcome but are not the sole factor, it may be useful to indicate in how many instances the output from the data matching program contributes to a result being achieved.

Further ideas on how to present cost and benefit information for data matching programs that have quantifiable non-financial benefits are presented above, under the heading ‘Methods of presenting costs/benefit information’.

Footnotes

[1] See www.comlaw.gov.au/Series/C2004A03712.

[2] See www.comlaw.gov.au/Details/C2010A00052.

[3] See www.comlaw.gov.au/Series/C2004A02562.

[4] See www.comlaw.gov.au/Details/C2011C00503.

[5] See www.comlaw.gov.au/Details/C2006C00591.

[6] Of the programs administered by the Department of Human Services, only the Centrelink Master Program participates as an assistance agency.

[7] Section 3 of the Data Matching Program (Assistance and Tax) Act 1990 (Cth) provides that ’assistance agency’ means:

  1. the Department of Health and Family Services; or
  2. the Department of Employment, Education and Training; or
  3. the Department of Social Security; or
  4. the Department of Veterans’ Affairs; or
  5. the Human Services Department.

[8] See Guidelines for the Conduct of the Data-Matching Program.

[9] See www.comlaw.gov.au/Series/F2008L00706.

[10] See www.comlaw.gov.au/Series/C1953A00095.

[11] Data-mining has been defined as ‘a set of automated techniques used to extract buried or previously unknown pieces of information from large databases’ (Information and Privacy Commissioner Ontario, Data Mining: Staking a Claim on Your Privacy (1998), 4).

[12] Data analytics (DA) involves processes and activities designed to obtain and evaluate data to extract useful information. The results of DA may be used to identify areas of key risk, fraud, errors or misuse; improve business efficiencies; verify process effectiveness; and influence business decisions.’ (ISACA , formerly the Information Systems Audit and Control Association, 2011, Data Analytics – A Practical Approach, available at www.isaca.org/Knowledge-Center/Research/ResearchDeliverables/Pages/Data-Analytics-A-Practical-Approach.aspx )

[13] See Privacy Act, s 40(1).

[14] See Privacy Act, s 52.

[15] See Privacy Act, s 40(2).

[16] See Privacy Act, s 28B(1)(b).

[17] See Privacy Act, s 33C.

[18] See Privacy Act, s 28A(2)(d).

[19] See Privacy Act, s 28A(2)(b).

[20] See Privacy Act, s 32.

[21] See Guide to Undertaking Privacy Impact Assessments.

[22] See FOI Act, s 8(2)(j), s 8A.

[23] See Part 13 — Information publication scheme

[24] See www.ag.gov.au/RightsAndProtections/IdentitySecurity/Documents/
Data%20matching%20better%20practice%20guidelines%20[PDF%20775KB].pdf. Document no longer available by this link.

[25] See www.naa.gov.au/Images/GDA24_tcm16-47735.pdf.

[26] See Privacy Act, Part VI.

[27] See www.finance.gov.au/publications/finance-circulars/2006/docs/Intro_to_CB_analysis.pdf. Document no longer available by this link.

[28] Available from the Australian Government Publishing Service. See http://trove.nla.gov.au/work/20627693?q&sort=holdings+desc&_=1373421533243&versionId=25458841.

[29] See www.finance.gov.au/publications/finance-circulars/2006/docs/Handbook_of_CB_analysis.pdf. Document no longer available by this link.