The Statistics of Anomaly Detection

2025-02-25T09:56:54-05:00

If you collect any kind of real-time data, you're probably interested getting alerted when this data goes out of whack, but setting static thresholds has the downside of having a lot of false positives. Using Anomaly Detection can help determine normal patterns and alert you when those go out of whack. In this talk we'll look at some of the methods of detecting anomalies in your data. https://confoo.ca/en/2025/session/the-statistics-of-anomaly-detection Leave feedback at https://confoo.ca/en/2025/feedback/DB5044AB426889A6A2CA1510FBB56617

Core Web Vitals and their effect on User Experience

2025-02-25T09:54:28-05:00

The Core Web Vitals - Largest Contentful Paint, Interaction to Next Paint, and Cumulative Layout Shift - were chosen because they are a close proxy for how good or bad a user experience is. There are some things to be aware of... Are they available on all page types, or across browsers, when do they become available, how do we capture them, etc. In this talk we'll understand what the Core Web Vitals are and how to make use of them. https://confoo.ca/en/2025/session/core-web-vitals-and-their-effect-on-user-experience Leave feedback at https://confoo.ca/en/2025/feedback/D0D5DA589D54778514141144094BA3C0

Measuring & Analyzing Core Web Vitals

2024-10-24T19:20:33-04:00

Migrating from Julia 0.4 to 1.6

2024-07-12T06:06:19-04:00

When Julia 1.0 was released several years ago, we were faced with the challenge of migrating over our large Julia 0.4 codebase while making sure our customers didn't notice a thing. This talk is a story about reassessing life choices, keeping customers happy, and a quick build, slow release, 5 year, 40 step migration from 0.4 to 1.6. We hope to never have to do this again! We started our Julia journey with version 0.2 over 10 years ago, and upgraded fairly quickly up to 0.4.7 as new versions were released. Then we started getting more customers, and our APIs stabilised, and migrations became expensive, so we sat at 0.4.7, watching every quarter as various dependencies reached the end of their supported lives. Given that migrations were expensive, we knew that whenever we migrated, it we'd have to make it count for many years to come. We started migrating our code to 1.0, deciding to skip all the advice about migrating to 0.7 first. While this was happening, Julia moved to 1.1, 1.2, 1.3, 1.4, and 1.5, so we decided to jump to 1.5 instead, and then Julia released 1.6. Over the course of the migration we developed a series of 40 steps that allowed us to change the underlying library while requiring only minimal changes to our customers' code. In this talk, we will quickly go over some of the more surprising changes we've had to make. We will not discuss plans for 1.10 or 1.11.

Using Julia in an Anomaly Detection Pipeline (JuliaCon 2024)

2024-07-10T19:04:52-04:00

Julia is a great multi-purpose language, but it also fits in as a component in a larger multi-language ecosystem. At Akamai, we use Julia as part of our data pipeline to do Anomaly Detection and Alerting on web performance data. In this talk, I'll cover the tasks delegated to Julia as well as how it fits into the rest of our development and operations stack. Julia is very good at running data analysis on columnar data and temporal data, ie, the kind of data we have a lot of where I work. We have packages to do regression analysis, hypothesis testing, signal processing, and more, allowing our development team to focus on business logic and data pipelines. In this talk, we'll cover how our Data Scientists use Julia to analyze data, and develop algorithms that can then be operationalized into a real time data pipeline and we'll see how Julia complements a Java based web application that handles data collection and alerting.

Curling with Julia (JuliaCon 2024)

2024-07-10T04:26:24-04:00

We have a Julia application that communicates with other services over REST APIs. These services use mutual TLS for authentication, which in turn requires the use of client side TLS certificates. We found that LibCURL was the best way to do HTTPS with client certificates, but with a very C like interface, it was hard for Julia developers to use. This led us to build CurlHTTP, which is a very Julia like interface for doing HTTP(S) with LibCURL. LibCURL.jl and HTTP.jl are the two primary ways to write HTTP clients in Julia. They both have their pros and cons. HTTP.jl While HTTP.jl has a very easy to use interface to access the majority of features, including cookie handling and streaming, it suddenly becomes very complicated when you try to use TLS client certificates. HTTP.jl also doesn't have the ability to do multiple downloads in parallel on a single execution thread. LibCURL.jl LibCURL makes it very easy to do mutual TLS, supporting client and server TLS certificates with well documented APIs, and has a well documented multiple download interface, but it provides a fairly low level API, and is primarily suited to developers familiar with writing applications in C. For our use case, we decided to build a higher level Julia wrapper around LibCURL's HTTP functionality (similar to what SMTPClient.jl does for SMTP over LibCURL), while making some of curl's more complicated features easy to use in a Julian way. CurlHTTP.jl The result is CurlHTTP.jl, which provides a high level HTTP interface to LibCURL, supporting single and multiple downloads, mutual TLS, and data streaming. Talk Structure This talk will first briefly cover the features of CurlHTTP and how to use it. We will then spend a little more time going into some of the nuances and gotchas we faced while developing this library, particularly with passing Julian data types through the C interface and back to our callbacks, and with dealing with memory management in a clean way.

Curling with Julia

2024-04-02T20:19:51-04:00

Julia already has LibCURL packaged with it, and has all the packages behind JuliaWeb to do HTTP, so why did we need another HTTP package built on LibCURL? In this talk I go over our motivation for writing CurlHTTP, and talk about some of the cool features that differentiate it from other libraries available in Jula. We’ll look at the CurlEasy and CurlMulti interfaces and show how easy it is to do things like mutual TLS (mTLS) authentication. CurlHTTP is in the Package Registry and is available on github at https://github.com/bluesmoon/CurlHTTP.jl

We Love Speed: Understanding Cognitive Biases in Performance Measurement

2023-05-11T13:38:18-04:00

When measuring web performance, we often try to get a single number that we can trend over time. This may be the median page load time, hero image time, page speed score, or core web vitals score. But is it really that simple? Users seldom visit just a single page on a site, so how do we account for varying performance across multiple pages? How do we tell which page’s performance impacts the overall user experience? How do various cognitive biases affect the user’s perception of our site’s performance? As developers and data analysts, we have our own biases that affect how we look at the data and which problems we end up trying to solve. Often our measurements themselves may be affected by our confirmation bias. This talk is targeted at individuals who want to understand the business impact of their site’s performance, and how biases in data can affect that. In this talk, we’ll go into different biases that may affect user perception as well as our ability to measure that perception, and ways in which to identify if our data exhibits these patterns. References: Ebbinghaus, Hermann (1913). On memory: A contribution to experimental psychology Kahneman, Daniel (2000). "Evaluation by moments, past and future" Baumeister, Roy F.; Finkenauer, Catrin; Vohs, Kathleen D. (2001). "Bad is stronger than good" Staw, Barry M. (1997). "The escalation of commitment: An update and appraisal" Arkes, Hal R.; Ayton, Peter (1999). "The sunk cost and Concorde effects" The impact of network speed on emotional engagement Ericsson ConsumerLab neuro research 2015 Wikipedia Paper on User Satisfaction v/s Performance Toward a more civilized design: studying the effects of computers that apologize The fastest way to pinpoint frustrating user experiences

1000 JavaScript errors are as good as None

2022-10-25T11:43:33-04:00

Does your site’s webdev console show thousands of errors on a user’s browser but none on yours? Do you receive complaints from users that they’re unable to complete a purchase and you can’t reproduce it on your end? When studying JavaScript errors, we can easily get lost in a very large number of errors that all seem to say the same thing. How do we tell if different errors really refer to the same root cause? How do we tell which errors are the most important to address? If we had to pick just one error to fix, how do we tell which one has the most impact on our users and business? In this talk, we look at ways to de-duplicate JavaScript errors across browsers and devices, and then how to identify which ones affect our users the most. Do some errors block the page from rendering? Do other errors cause increased user frustration? Is there an error that prevents users from finishing a checkout flow? Presented at: iJS Munich

Understanding Cognitive Biases in Performance Measurement

2022-10-25T11:05:45-04:00

Understanding Emotion for Happy Users

2020-11-16T10:51:13-05:00

Blog Post: https://tech.bluesmoon.info/2020/11/understanding-emotion-for-happy-users.html There is little doubt that closely watched metrics tend to improve over time while metrics that are either uninteresting, or hard to collect tend to get ignored, and often get worse. Web developers and operations professionals have many tools available to understand and improve the overall performance and health of their websites. Tools that tell us how fast our site is, what the bounce, or conversion rate is, whether any links resulted in a 404, and more. All these tools focus on making development more efficient and reducing errors. When we focus only on improving raw numbers, we forget that there’s a human on the other end who can become frustrated with an overall bad experience and not just the numbers we’re looking at. In this talk we will look at how to collect, analyse, and act on a few metrics that tell us more about how our user feels when using the site. We can trend how these metrics change over a user’s browsing experience, and we can improve over time the metric that really matters: user happiness. Some of the metrics we’ll look at are: * Rage clicks, missed clicks, and dead clicks * Cursor thrashing * LD50 (median lethal dose) We’ll try and correlate these metrics with things that are easy to improve developmentally, like smoothness, jank, visible transitions, and more. At the end of this talk you will understand which metrics are useful to measure, how to measure them, and how to tie them in to your development goals.

Understanding Network Protocols & Server Configuration

2019-05-30T10:50:52-04:00

In the previous talk, we covered some easy wins with performance optimisation. In this talk, we’ll look at aspects of frontend performance that need to be tuned or fixed server-side, either via a web server, or operating system configuration. We’ll look at HTTP level questions like: * How does H2 prioritization work? * How does H2 push work? * What is QUIC (HTTP/3)? We’ll also look at TCP level questions like: * What are MTU, MSS, and TCP Windows and how do they affect performance? * What are TCP slow-start and fast-open? * How to use Wireshark to study network activity? At the end of this session, attendees will understand how to debug low network level performance problems, and configure systems for performance improvements.

Improving Frontend Performance

2019-05-29T11:02:12-04:00

There are many areas to work on when improving web performance. For every type of resource that makes up a page, there are different ways of optimising them. In this talk, we’ll look at several low hanging fruit – assets that can be optimised with very little work, most of which can be automated into your CI pipeline. We’ll then go into some more advanced front end topics that require development work, and humans to make a decision on whether it’s worth it or not. Topics that we will cover are: * Image Optimisation * Compression * Combining or Splitting CSS & JavaScript * Whether to use domain sharding or not * CSS & JavaScript considerations * Service Workers * Preload hints At the end of this talk, attendees will have the skills needed to bring about immediate performance improvements to their sites, often with very little effort.

Understanding Frontend Performance

2019-05-28T17:44:25-04:00

One of the first things to do when working on performance improvements is to start measuring. We cannot improve what we don’t measure, and things that are measured and watched tend to improve over time. The earlier talks in this track discussed which metrics were important to measure. In this talk, we’ll go over how to measure performance, and how to get useful diagnostics out of our tools. We will cover: * Tools to measure web performance * Understanding waterfalls * Understanding browser diagnostic data * Measuring RUM data * How do modern web applications complicate the situation? * Single Page Applications (SPAs) * Accelerated Mobile Pages (AMP

Metrics that Matter

2019-05-28T15:26:34-04:00

Whenever we speak about measuring web performance and user experience, we typically refer to static events in the browsing experience. Modern websites, however, are far from static, and user interactions with these websites have continuous aspects that cannot be represented by distinct events. * How do you measure user frustration? * How do you tell how fast your users expect your site to be? * How do you tell if users were frustrated or annoyed before they got to your site or if your site caused that frustration? * What are TTVR, TTI, TTFI, FID, and which of them is the most important? * How do you tell if your page locks up during user interaction? In this talk, we’ll look at some new user experience metrics. We’ll find out how to measure page responsiveness, smoothness, jank, and usability. We’ll learn about things like Rage Clicks, Missed Clicks, Dead Clicks, and Cursor Thrashing. We’ll also look at real user data that we’ve collected showing how these aspects of the page affect user behaviour.

The Human Aspect of Performance

2019-05-23T09:45:32-04:00

The performance of a website affects the users of that website in ways that are hard to measure. The lack of responsiveness, lag, jank, and UI lock ups can cause perceptual dissonance that break the user’s flow of thought. A page not responding in the way the user expects, or as fast as the user expects it to, can result in frustration and anger. From a business point of view, frustrated and angry users are unlikely to convert, and buy items from the site. Apart from the psychological and emotional effects of bad performance, the factors that result in bad performance can also make sites expensive to use. How much bandwidth does your site require to be downloaded? How much does that bandwidth cost in different parts of the world? How expensive is your site in terms of battery usage?

UX & Performance: Metrics that Matter

2018-10-09T11:15:04-04:00

In this talk, we’ll look at some of the new user experience metrics that the boomerang team has been collecting. We’ll find out how to measure page responsiveness, smoothness, jank, and usability. We’ll learn about things like Rage Clicks, Missed Clicks, and Dead Clicks. We’ll also look at real user data that we’ve collected showing how these aspects of the page affect user behaviour. JavaScript provides us with many hooks into measuring performance and user experience. Let’s learn to collect them and understand what to expect when we optimise for them.

Sciencing Data

2017-08-07T18:30:36-04:00

A short overview of what Data Science is and some of the kinds of insights we look for at Akamai. visual data like spice on a hot griddle patterns emerge