When people submit applications to Immigration, Refugees and Citizenship Canada they typically have spent significant time carefully completing forms and assembling documents. They expect that their applications will be processed by visa officers who carefully review the information before them. However, applicants need to understand that their artificial intelligence is playing an increasing role in visa processing, as is the bulk processing of applications.
Why this is a timely matter to discuss
IRCC has not been forthcoming with how it uses technology to process applications, however, through a series of Access to Information Act requests as well as Federal Court of Canada litigation the public is beginning to get a sense of measures being implemented.
Predicative Learning
Automated processing of some categories of applications is not new. Since 2015 most visa-exempt foreign nationals have had to apply for an Electronic Travel Authorization before they could board a plane to travel to Canada. These applications were for the most part automated applications.
What is less known is that in 2017 IRCC successfully conducted a pilot in which automated systems based on predicative analytics triaged and automatically approved low-risk online temporary resident visa applications from China. Visa applications were sorted into tiers – the lowest risk for auto-approval, medium and high risk for officer review. This triage model was deployed for all applications from China in 2018, and in the same year was piloted in India. The goal appears to be for artificial intelligence to automatically approve low-risk applications, with officers only manually assessing those that have been flagged as medium to high risk.
As of 2020 the analytics systems accelerate the eligibility approval of straight-forward cases. They do not refuse applications or recommend refusing applications (although presumably if a visa officer knows which tier an application is the officer might be implicitly swayed). Officers continue to perfrom admissibility assessments on all applications. The final decision is not made by the system.
From the system launch date to January 28, 2020, the approval rates for each tier were as follows:
Advanced Analytics
The history of the use of advanced analytics in IRCC decision making can be briefly summarized as follows. Much of the following is paraphrased from the ATIP below.
In the deployment of advanced analytics, two sets of rules are utilized. First, a set of automated triage rules was deployed in April 2018 to identify TRV e-Applications submitted from China that meet these rules. The automated triage rules, also known as the ‘Officer Rules’, were created by IRCC’s Beijing office using statistical information, industry trends, and historical data. Second, after the Officer Rules/triage rules are applied, an analytics model is utilized to identifyTRV e-Applications which meet rules established by data analytics software using industry best-practice methodology. ‘Model Rules’ are created by the SPSS Modeller software using historical data on prior processed TRV applications from China. It is important to understand these two terms.
The Machine Learning uses a decision-tree model, which can be graphically represented as splitting the dataset into a series of branch-like segments forming an inverted tree originating with a root node at the top (see Image 1 below).10 Each split, where the large data group is broken into progressively smaller groups by posing an either-or scenario, is referred to as a node. The bottom nodes of the decision tree are called leaves {or terminal nodes}. Each node represents a fact about an applicant and the leaves represent classes of acceptance or refusals. For each leaf, the decisoin rule provides a unique path for data to enter the class that is defined by the leaf. The leaves are mutually exclusive, i.e. they are designed so that no application oculd meet the conditions of more than one leaf.
Once data sets are identified, they are studied by the modeller in search of ‘features’ or ‘characteristics’, called variables that can be used to build the Model Rules. The variables/flags are the data fields used to find patterns and correlations and make reliable predictions of the outcome. From the various fields of information extracted from IRCC records, the analysts must determine which ones are relevant and contain reliable and consistent data for modeling purposes.
The data that are the flags/variables comprising the Model Dataset, including the outcome/decision, are used by the advanced analytics software to ‘train the algorithm’. The SPSS software tries millions of combinations of applicant characteristics (flags/variables) in search of patterns and to assess how they correlate with approval or refusal of a given application, i.e. the identified target variable. When it finds insightful combinations that repeat and reliably and consistently lead to the same outcome, it formulates these patterns as decision rules-the ‘Model Rules’. The reliability and consistency of a Rule leading to an outcome is referred to as the confidence threshold of the given rule.
The Officer Rules are created as a set of triage rules, which are applied prior to the Model Rule triaging.
The Model Rules are created using historical, personalized applications for each line of business and using analytics software to identify trends, patterns, and commonalities within those applications.
The following Privacy Impact Assessment provides a detailed overview of the development of these two “rules” and associated concerns.
In Luk v. Canada (Citizenship and Immigration), 2024 FC 623, Madam Justice Aylen held that the use of algorithms or artificial intelligence to process applications is not in of itself a breach of procedural fairness.
Untitled Extract PagesSpousal Sponsorships
In the family reunification context, from April – October 2021, IRCC ran a pilot triage model which created reports for officers for their use in making eligibility decisions. These reports are transitory and IRCC purges them after use. The model applies seven rules. Applications that are deemed low risk/low complexity are put in a Green Bin, while complex applications are triaged into a Standard Bin.
Spousal Sponsorship AIChinook
In addition to automated triaging IRCC has also introduced software so that officers can bulk process applications. The software tool is known as Chinook.
According to an affidavit that IRCC filed in Federal Court, Chinook is a standalone tool that streamlines administrative steps. Applicant information is extracted from their applications and presented in a spreadsheet. Visa officers are assigned a workload of applications through Chinook. They are able to see multiple applications at a time on a single spreadsheet. This allows them to review the contents of multiple applications on a single screen, and allows them to complete administrative steps through batch processes. It also allows visa officers to create “risk indicators” and “local word flags” so that officers can identify possible applications in the processing queue of concern or priority.
According to the Federal Court affidavit, when visa officers enter Chinook a message pops up which says, amongst other things, “The Chinook User Interface allows you to view multiple applications for review and initial assessment. It does not replace reviewing documents.. and/or reviewing other information… The refusal notes generator is meant to assist with general bona fide refusals. If the notes do not reflect your refusal reasons, please write an individual note.”
The Chinook user guide can be found here:
Chinook User GuideConcerns
There have been many concerns raised about the implementation of automated triaging and Chinook. These include the possibility that it is what has led to increased refusal rates, that individual care is not being given to applications, that applications are not being carefully reviewed and instead quickly bulk refused, that AI flagging a file as high-risk will lead to an officer wanting to simply affirm the AI’s finding, that refusal reasons are increasingly consisting of boiler plate templates which is not helpful for applicants, and that it may perpetuate systemic racism.
Because IRCC has not been transparent about the implementation of these systems and their results it is difficult to confirm if these concerns are founded. Regardless, it is important that those submitting applications understand that Canada’s immigration system is no longer one in which human officers meticulously process individual applications in the order that they are received. I have previously written about how it is important for individuals with refused applications to obtain the internal reasons for refusal, or Global Case Management System (“GCMS”) notes. IRCC’s use of artificial intelligence and bulk refusal generators makes this even more important, as a review of the internal reasons or GCMS often indicative of whether such software was used, and whether a refused applicant should either file a reconsideration request or seek judicial review to see if a human may reach a different conclusion.
Analytics Based TriageAI
Advanced Analytics