AI, Applied

What changed. What matters. What to do about it.

Issue 06  ·  Fortnightly  ·  5-minute read

New privacy rules are coming. Do you know which decisions your software is helping to make?

A recruitment tool ranks your job applicants. Software decides whether a customer is approved for credit. A spreadsheet formula scores which clients get help first. These processes can already exist inside everyday business systems, often without anyone thinking of them as automated decision-making.

Most leaders can name the AI tools their business uses. Far fewer can say where software — AI or not — influences a decision about a customer, applicant or employee. That gap is the real leadership question here. Those are the decisions that affect people, so they're the ones you most need to be able to explain.

Part of this is now going into law. From 10 December, new Australian Privacy Principles (APP 1.7–1.9) take effect. They apply to organisations covered by the Privacy Act that arrange for a computer program to make a decision, or to do something substantially and directly related to making one, where the decision could reasonably be expected to significantly affect someone's rights or interests and the program uses their personal information. In those cases, the organisation's privacy policy must describe the kinds of personal information used and the kinds of decisions involved.

It's a disclosure requirement, not a ban, and it covers only qualifying decisions — not all software use. These requirements are law, under the Privacy and Other Legislation Amendment Act 2024. The Office of the Australian Information Commissioner (OAIC) published its guidance on 30 September. That guidance explains how the regulator reads the law; it isn't law itself.

Source (law): Privacy and Other Legislation Amendment Act 2024 →

What counts? Three things must all be true: a program makes, or plays a key part in, a decision; the decision could reasonably be expected to significantly affect someone's rights or interests; and it uses personal information about them. The Act gives examples such as decisions that affect rights under a contract, or access to a significant service or support. The OAIC reads “significantly” as more than trivial, and says “computer program” includes rule-based processes, AI and machine learning, ordinary software and apps, and generative AI. Its worked examples include a spreadsheet formula that ranks clients for health and aged-care services. It lists recruitment software that sorts candidates, loan and credit approvals, insurance eligibility and personalised pricing of significant goods as decisions it would generally treat as in scope.

A human signing off doesn't automatically take a tool out of scope. The OAIC says a program can still count where it is “a key factor in facilitating the human's decision making”, and it generally expects AI outputs used this way to be in scope unless there is extensive human oversight and control. Software that doesn't use personal information, or whose output doesn't significantly affect anyone, falls outside the rule.

Source (OAIC guidance): Fact sheet →

Who's covered? The rules apply to “APP entities”. For a business, that means an organisation covered by the Privacy Act: generally one with annual turnover above $3 million, plus some smaller ones, such as health service providers and businesses that trade in personal information. The OAIC generally expects responsibility to stay with the business using the information, not the software supplier, although suppliers should give you high-level information about how their products make decisions. Trade secrets don't have to be disclosed; the kinds of data and decisions do.

Source (OAIC): Who the Act applies to →  ·  Source (law): Privacy Act 1988 →

Staff decisions are less clear. The OAIC's examples include AI reports that rank employee performance or decide bonuses. But the Privacy Act also exempts certain employer handling of current and former employees' records, and the OAIC's fact sheet doesn't say how the two fit together. That exemption generally doesn't extend to job applicants. Take advice before assuming either way.

Source (OAIC): Employee records exemption →  ·  Source (law): Privacy Act 1988 →

From 10 December, some organisations will have to describe those decisions in their privacy policy. The better position is knowing them first.

That's it for this edition — short on purpose. If one of these changes what you do this week, it did its job.

AI, Applied is an executive briefing for leaders who don't have time to follow AI but need to know what it means for their business.

AI transparency: AI tools assisted with research and drafting. Final selection, analysis and editorial review by Mia Pursehouse.

Read past issues  ·  Manage preferences  ·  Unsubscribe

Mia Pursehouse, Buttaba, NSW 2283

Sent to Reader