Learning campaigns

The status quo

CybSafe launched in 2018 as B2B security awareness and training (SA&T) platform. It offered a library of modules to its customers, each covering a specific cyber security topic. Admins could assign these to groups of users as they saw fit. The assumption was that knowledge and awareness of cyber security enables people to protect themselves and their organisations from cyber threats.

‘Security awareness training is dead’

At the same time, CybSafe’s vision had changed. Driven by the Science & Research team, focus shifted away from SA&T to Human Risk Management (HRM). It acknowledges that several factors influence human cyber risk. Knowledge & understanding is just one of them.

We jumped the gun...

In the interest of speed, the Product Manager and I went straight to the drawing board. The idea that gained the most traction was called smart groups. Users would be added and removed from groups based on shared attributes. Because modules are assigned to groups, users would be automatically assigned those modules as well.

Workshop time

I decided to take a step back and ran an ideation workshop with the squad to gather and consider a range of ideas. Participants included:

• 1 Product Manager
• 4 Engineers
• 1 Customer Support Manager
• 1 Behavioural Scientist

After everyone had added their ideas, we combined any duplicates and mapped them to a impact-effort matrix.

Back on course 😏

As we discussed the highest impact ideas, three of the bigger bets came together to form a new concept called Learning campaigns (I wanted to just call them Courses). We concluded that the high effort required was necessary to build an effective solution. There was also leeway, as other squads had already worked on similar capabilities that we could reuse. As for the quick-wins, they were done in a hack-day!

Task flow

Having figured out the basic structure of a single learning campaign, I organised its components into a task flow to determine the sequence of steps that would be necessary for admins to create one. It also facilitated discussion with the squad about how much work would required and what additional components were necessary for campaigns to function.

Learning campaigns

Problem: Admins could only assign one set of modules to each group of users. This meant running multiple training initiatives at once involved a tonne of manual micro-management.

Solution: Learning campaigns are collections of modules that admins can create and enrol users into. It is now much easier to organise and run multiple training initiatives based on different topics and objectives.

Audience rules

Problem: The only way to assign a training module to a user was manually via groups that were often based on departments. This made it almost impossible for admins to deliver truly personalised training to users.

Solution: Audience rules automatically enrol users into campaigns based on their objective actions or attributes. For example, users who frequently fall for simulated phishing emails can be enrolled into a campaign that focuses on techniques to spot phishing emails.

Modules

Problem: Admins found it difficult to find where to assign modules to users because it was via groups, a place they did not expect it to be.

Solution: Module assignment was now done via Learning campaigns in the Learning part of the product.

Schedule

Problem: Users received all modules at once, overwhelming them and preventing admins from running initiatives based on certain events or the time of year.

Solution: Campaigns can be scheduled with specific dates or run continuously, allowing phased training and enabling seasonal initiatives.

Usability testing 🫷Releasing to production

I wanted to conduct usability tests on the designs to identify any issues in the new experience before the engineers started building it. But the emphasis on quick delivery meant we were pressed for time. As result, the squad was keen to start development. After a little persuasion from both sides, we met in the middle and we found a good alternative.

Releasing to production 🫷Beta testing the MVP

The CybSafe platform lets customers opt-in to beta test new features. So we agreed to release Learning campaigns to beta testers first. We would then ask them for feedback once they had time to test out the new feature. This would reveal the most serious issues to us, allowing us to fix them.

There's a template for that

Problem: Admins were overwhelmed by the number of options when setting up a campaign. But this complexity was required for campaigns to function and be value to customers.

Solution: Templates are a set of pre-made campaigns with everything already setup. All admins have to do is choose what they want to use.

No more robo speak

Problem: The audience rules used programming-like language (if, else, etc.) to show more complex options. But this confused admins who lacked programming experience.

Solution: Audience rules now use plain, user-friendly language, making them easy for anyone to understand, regardless of technical expertise.

Options now in the right place

Problem: Admins often missed the ‘Activate’ switch in the first step of the campaign flow. This left them wondering why their users weren’t receiving modules after a campaign was created.

Solution: The activate switch was a ‘simpler’ compromise to let admins ‘save’ their progress. So we replaced the switch with the ‘Save draft’ button that I originally proposed. Technical constraints prevented autosaving.

Notifications

Problem: Admins wanted to be able to notify users when they had been enrolled into a campaign, as well as send them reminders. While they could do this with the ‘Nudge’ feature, there wasn’t a clear journey to do so from Learning campaigns.

Solution: I planned to add a widget to the campaigns page to guide admins to the nudge feature and setup the notifications there. But this was out-of-scope due to time and ownership constraints. Instead we added a ‘Notifications’ step that let admins select from three pre-made notifications.