Barclays Sense

Fraud Machine Learning Platform

 

  • At the beginning of 2018 Barclays decided to invest heavily in emerging technologies and improve its in-house technical capabilities. I was part of an ambitious project to create a fraud machine-learning platform.

    Barclays Fraud Machine Learning platform called Sense detects fraud by employing artificial "Experts" that leverage machine-learning algorithms to analyze patterns and trends in real-time and uses data-driven strategies to make intelligent decisions.

    To comply with my non-disclosure agreement, I have omitted and obfuscated confidential information in this case study. All information in this case study is my own and does not necessarily reflect the views of Barclays.

  • Our goal for the project was to improve the usability and information architecture of Barclays Sense. However, after some initial user research, we realized that the application has to grow into a platform that embraces a rapidly evolving business and more diverse business groups.‍

  • As a UX Architect on the fraud transaction cycle, I provided the strategic direction to create a product that is grounded in well-researched user and customer needs. .

Research

From the onset, I wanted to understand the current technology landscape and the problems faced by the users. We have conducted several rounds of contextual inquiries with the Fraud strategists and data scientists. They gave us an in-depth walk-through of existing products like INSTINCT & TSYS etc. that are being used. These interviews gave us enough information about the user expectations and the pain points.

Here are the key research insights:

  • The user experience of these applications is very cumbersome and doesn’t account for the user’s natural mental model.

  • The workflow for the users is not completely automated.

  • Some of the technologies and backend services that are used are old and are very slow.

  • The users are spending almost 40% of their time on manual processes involving documentation and red tape.

  • There is a lot of interdependency between systems to push a set of fraud rules and it takes a long time to deploy. This defeats the idea of quickly reacting to the fraud as it happens.

We learned after talking to the different stakeholders is that Fraud is very dynamic and fighting fraud in real-time is very important. The current solutions in place are not as effective. This expanded the scope of the project. We started with the idea that the scope will be limited to managing fraud rules and rulesets, but it became a complex system that handles a lot of other components related to fraud detection.

 

Major Pain points

Some of the major pain points with the existing design.

Old audit trail screen

Old ruleset screen

 

Improper Interactions

User are frustrated that the interactions on the applications are unexpected and doesn't feel natural.

Information Architecture

The information hierarchy of the application doesn't match the functionality required by the business.

Barclays Brand

The business would like to get the application match the Barclays brand guidelines and standards.

Responsive Application

The users complained that the design is not responsive and doesn't render properly on their Surface Pro Tablet.

Reduce Manual Processes

One of the biggest complaints the users had was that the existing applications forces them to maintain a lots of spreadsheets and documents that facilitate collaboration and backup during audit. They wanted the redesigned application to remove this dependency on manual processes.

Optimize the Model Flow

The creation, testing and deployment of models is very cumbersome and involves a long process. The business would like to optimize this process.

Personas

We recognized that there were 3 key user types that our product. Fraud strategists will be the primary user group.

  • Fraud strategists

  • Data scientists

  • Engineers

The redesign

After conducting whiteboarding and sketching sessions, we have agreed on the general information architecture and a framework that will guide our design decisions:

  • The application should support different business groups

  • Separation of duties is very important

  • There are some components developed by engineering that have to be used in the redesign

 

The Hub

The screen shows information based on the user’s profile. A simple clean UI that focuses on progressive disclosure. The user starts with a domain and proceeds in a linear path.

 

Domain Landing Screen

This screen shows rulesets, approval requests, and deployment request tabs based on the roles assigned to the user. Separation of duties is very important to maintaining security.

Used a simple table with filters and ellipses icon to see actions. This table pattern is used across the application.

 

Editing the Ruleset

This is the most important screen in the application where users are allowed to write rules, test rules, update machine learning models, etc. and validate them.

 

Approval Requests Screen

The users are allowed to approve the rules that are authored by other users.

 

Deployer Screen

A deployer is allowed to deploy the rulesets that are not authored or approved by them.

I have intentionally omitted some confidential flows here.

 

Comparing Rulesets

The authors, approvers, and deployers can compare two rulesets to understand the differences before making a decision.

Color-coded and displayed the change type in parenthesis for updated, moved, new, and deleted rules for better findability.

Usability Testing

We have conducted usability testing with 5 participants and collected feedback. We have identified 5 major changes and 4 minor changes which will be implemented as part of the next release. During the test, the users rated the ease of task completion for different tasks on a scale of 1 to 7 where 1 being very difficult and 7 being very easy. The average ease of task completing is 6, which indicates the users were very comfortable using the application.

Also, a SUS survey was completed by the different users and the score was 77.0 which is above the general average of 68.0. But there is a lot of scope for improvement in the application in terms of navigation, information architecture, page content, messaging etc.

Result

  • Launched in September 2021 and it handles 14 million transactions per day.

  • Saving Barclays £30 million/year

  • SENSE was crowned the best
 application in group technology awards 2021

Prototype

Would you like to view a clickable prototype?

View Prototype