Algorithms for Hiring, Credit..What Next? Perhaps Caution Should be Exercised

by Stacey Lett, Regional Director of Operations – Eastern U.S. – Proactive Technologies, Inc.

We are pushed from all sides to embrace advancing technology meant to impact every aspect of our lives. Peer pressure – from friends, family, colleagues, industry “experts” drive us to consider embracing “our future” – often explaining away the disruptions it causes to our present. Sometimes naïve, but always enthusiastic, media compete to be the first to break the news, bombard us with everything from subtle shaming to industry-driven pushes to accept and use technology – even if lacking thorough testing or proper consideration of all ramifications from its usage. If the technology causes damage, shoulders are shrugged and the horizon is scanned for the next.

Driven by massive amounts of marketing cash –often to create the illusion of trends when market acceptance is tepid – who is helped and who is hurt by the innovation is a distant afterthought. The damage can be done and those promoting the technology push to broaden its acceptance. If press coverage is too harsh when its promise comes up short and shareholder interest wanes, abandon it and move on to fabricate the next “trend.”

Take, for example, the recent examples of Apple and Goldman Sachs credit cards, which it appears  issued lower credit limits to spouses of husbands who shared the same credit score. “A tech entrepreneur, David Heinemeier Hansson, first raised the issue when he tweeted that the Apple Card’s algorithms discriminated against his wife, giving him 20 times the credit limit it had given to her.” His wife’s credit score was much better than his own. This, also, happened to the spouses of Apple cofounder Steve Wozniak and other prominent figures.

Some banking algorithms utilize around 400 attributes in its determination of creditworthiness. And that might be made up of many we wouldn’t expect, but we will never know since it is considered a “trade secret.” But, as in most cases of machine learning, a lot of trust and faith is afforded the technological advancement which is only rocked when something tragic, abnormal or, in this case anti-social and potentially illegal, is discovered.

Wired editor-in-chief Nick Thompson said in an interview, “My belief isn’t there was some nefarious person wanting to discriminate. But that doesn’t matter. How do you know there isn’t an issue with the machine-learning algo when no one can explain how this decision was made?” He went on to issue a warning that undetected bias in algorithms is “one of the biggest problems of our time right now” but the Goldman credit card story is “not the best example of this huge problem.”

Algorithms and AI have also made its way into online dating applications, insurance policy acceptance decisions and, more astonishingly, the hiring of employees. Employers who use a third-party applicant screening service may be unwittingly turning down potentially experienced and highly skilled workers on the basis of, well, who knows? Was it in consideration of a “Bona Fide Occupational Qualification” as required by Equal Employment Opportunity Commission regulations? A case in court may be needed to expose any bias and inadequacy to determine if that was the case.

We saw suspicious criteria creep into the hiring process over the last three decades. First, with the advent of resume scanners and key word search to screen applicants for a match with key words defined by the employer or third party service – words which were often not part of the applicant’s natural vocabulary. This technology was borrowed from spy agencies, who used it to read and analyze publications, correspondence and legal documents from around the word to uncover espionage plots.

Next, psychometric criteria doubled down on key word search concept with the addition of questionably non-job relevant criteria such as credit score, public records, missteps on social media and the length of previous employment. These types of attributes were toxic to the economic recovery of millions of American workers after the Crash of 2008. Rating an employment applicant on how well they survived an economic collapse that they had no part in is, just on its face, impractical, unethical and potentially illegal.

Artificial Intelligence (“AI”) has been promoted as the solution to the inadequacies, inaccuracies and inappropriateness of key word and psychometric screening…that is until it is understood and the detrimental effects of its usage in employment decisions is revealed as in the case of the extension of credit. The depth and scope of the damage could be much greater, and the irreversible effects it could cause could be more severe before a problem is discovered. If a person loses a job or is not hired due to questionable reasoning, their credit may be impacted while months go by searching. If their credit is impacted, it may be harder still to find a job. We should all approach AI application in employment carefully and more cynically than ever before.

The amount of data that exists on every one of us – unverifiable since we do not know who collected it, who bought it, or what was collected and have no way to correct it if inaccurate or untrue – is enormous. We add to our profile with every call we answer, email or text we make, every internet search we make, every online survey we complete, every app we download,  every purchase we make and with every Facebook post we make, read or “like.” We add to it when we visit a doctor or pharmacy, search the internet on physical symptoms or the side effects of prescriptions we fill. This data is collected, resold and consolidated and then used in these algorithms to decide if we are job worthy, credit worthy, insurance worthy…in ways only well-funded start-ups can imagine. All of this done without our permission, compensation or control.

We do not know, yet, how this data prohibits our upward mobility in society and the economy. Today in the United States, the loss of a job at an inopportune time, a medical emergency that wipes out a family, or the loss of a home or car due to the effects of a job loss may derail any career and leave one on the wrong side of the tracks for some time with little access to resources or road map to return. What’s worse is that very little of this data is relevant to why someone is qualified for a job. And without a job, a credit score is bound to worsen, which will make it difficult to get reasonably affordable health, car and home insurance or even access to an apartment…and the next job!

This could all be even worse. For a look at what the extreme of big data and the promise and perils of artificial intelligence look like, a PBS Frontline entitled “In the Age of AI, ”explored fears about work and privacy and traced a new industrial revolution that will reshape and disrupt our lives, our jobs and our world, and allow the emergence of the surveillance society.” AI proved to be an effective tool against parts of a society, in this case the Uyghurs of China.

Hopefully, as more information emerges about the dangers of AI and lack of government oversight, citizens,  government and industry leaders will put the brakes on unsupervised collection and use of data that should never be used for making selections that impact lives in such as sweeping way. For one sector of society to make billions of dollars collecting data without tacit, explicit approval, then make billions selling and using that data against the people who unknowingly, unwittingly and/or unwillingly provided it seems, at a minimum, to be unethical and against privacy laws at a minimum.

European citizens stood up and demanded laws to limit what data could be collected, how long it could be kept and measures to remove data from search engines with its “right to be forgotten” legislation. And this trend seems not ready to stop there.

If AI use in employment screening is allowed to grow and proliferate, this could make finding employable workers as difficult as it was to find credit worthy borrowers after the Crash of 2008 when half of the country’s credit scores were impacted. If your firm would rather use job-relevant, legally defensible job criteria to select job candidates and train workers to master the work you need done, and have shed your fear of even looking at other solutions, check out Proactive Technologies’ structured on-the-job training system approach for the accelerated transfer of expertise^TM to see how it might work at your firm, your facilities or your region. Contact a Proactive Technologies representative today to schedule a GoToMeeting videoconference briefing to your computer. This can be followed up with an onsite presentation for you and your colleagues. A 13-minute promo briefing is available at the Proactive Technologies website and provides an overview to get you started and to help you explain it to your staff. As always, onsite present