Koderama

6 things to know before autoscaling your Kubernetes cluster

2023-07-02T00:00:00+00:00

Have you decided to stop wasting money on nodes you don’t use? Do you want a flexible infrastructure to respond to real-time demand without manually setting your nodes? That is where Kubernetes autoscaler comes into the scene.

What is cluster autoscaler

When a pod ~~begs~~ requests Kubernetes to be scheduled, Kubernetes checks whether it has nodes with enough resources to accommodate the pod (based on the resources requested by the sum of all its current pods); in case it doesn’t, lower priority pods can be evicted, to accommodate the new pods.

If it can’t find a place for the new pod, it marks the pod in a Pending state and start the process to spin up a new node; the new node is provisioned and then can receive the new pods.

Here, I’ll describe six essential things to remember when preparing your cluster to scale smoothly, so you can go to bed without worrying about being paged at 3 am.

Resources definition Kubernetes likes to know about their workload beforehand; pods with neither requests nor limits defined run in the QoS(Quality of Service) class BestEffort, which puts them on the top of the eviction list. Make sure you have your pods with adequate quality of service, and please, don’t let your pods without their resources defined; autoscaler won’t get along with them well. You can use a policy engine like Kyverno or OPA to enforce it.

Watch out for Pods that Need Longer Periods for Graceful Shutdown: When dealing with these slow dudes, you have to make sure your autoscaler is patient and gives these pods the time they need to shut down gracefully; this can lead to slower scaling and will make you reconsider or rearchitect how those pods work.
Monitor properly Keep an eye on the speed at which your nodes are spinning up, how long your pods are scheduled end-to-end, the number of failed schedulings, and failures to spin up nodes. You don’t want those things to go unnoticed.
Balance of Nodes in Different AZs Don’t let your nodes spin on a single AZ; make sure to have some auto-balancing in place, clusterautoscaler has a flag exactly for this purpose. Make sure you don’t have any conflicts operations; for example, AWS has its own AZ Rebalancer and when operating along with Kubernetes autoscaler it can have undesired effects.
Select the best expansion strategy With the --expander flag, you can select among a few different strategies to select the node group to expand when needing to scale, least-waste is a good starting point, but you can also select based on price or priority.
Do you need a faster scale-up? Sometimes the time to spin up a new node can take a few minutes, which might not be optimal for some scenarios; you can achieve that by overprovisioning, basically running dummy pods with low priority, they will be evicted in case new pods with higher priority need to be scheduled, and the new pause containers will force the cluster to scale up!

Now, you can grab a coffee, open your cluster dashboard and look at all the magic of scheduling and scaling.

The confusion about Kubernetes requests

2022-10-19T00:00:00+00:00

There is a famous quote in computer science:

“There are only two hard things in Computer Science: cache invalidation and naming things.”

Name things is hard because you need to consider the function of the thing you are naming, not only in the present but also in the future, and yet to predict all possible confusions that might arise in people’s minds.

Recently, working on a project that required bringing awareness about resource utilization on a Kubernetes cluster, I noticed a common theme for confusion: Kubernetes resources requests and resources limits.

After analyzing ~~hundreds~~ three messages, I came to the conclusion that the confusion has a clear culprit: the bad naming for requests, limits is not that bad, requests on the other hand.

Requests is an extremely generic term, and everyone knows that the rule #1 of naming is: DO NOT use a generic name. Requests can mean anything, and the context of resources adds absolutely zero to the clarification.

I’m sure all the confusion would have gone away if, instead of requests, the Kubernetes team had picked a name that fits the function better, like guaranteed or reserved, because that is what it does, reserve or guarantees the resources specified there.

Then instead of resources requests, it would be resources reserved, it looks much clearer now, isn’t it? People would see and immediately understand that this is the amount reserved for their pod, and there is no more need to decode what the hell requests mean in this context.

But now it is too late; there’s no going back. We will need to live with it until the end of the days, or until some Kubernetes breaking change; who knows?

So next time you need to understand it, just replace requests with reserved; that is the amount of resources (CPU/Memory) your pod will have reserved, guaranteed, and the limit is the amount it can’t exceed.

Doesn’t it make it easier?

How Scrum has killed Agile

2022-10-06T00:00:00+00:00

In the beginning, there was ~~darkness~~ the waterfall, and it was bad, terribly bad. For some reason, we thought building software was the same as building an actual building, so we ended up with a pile of papers, endless meetings, stiff processes, and no software delivered.

In that dark times, the fear of change was constant; whenever a change was introduced in the software development cycle, it meant going back to square one, restarting long meetings, rewriting documentation, and getting approvals.

Later, IBM started pushing RUP (Rational Unified Process), a sort of iterative waterfall, which replaces the long cycles of the waterfall with shorter ones, a step in the right direction some would say, even though many companies just ignored the iterative part.

But corporate had metrics and a false sense of control, so they were happy, or they thought so. In the first years of the ’00s, a new hope joined the game; the Agile Manifesto was published, a simple, concise manifesto with clear sentences showing what software development should focus on:

Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan

It embraced the nature of software development as a process in constant evolution instead of requiring a rigid plan.

It was music to our ears, and I couldn’t wait to put into practice what they proposed, along with some techniques from Extreme Programming (TDD, pair programming). So, I decided to pitch it to the management of my company at that time, and their response was something along the lines of “We have built software for fifteen years; we know how to do it, and this Agile thing is only for small companies.”

A bummer, but only for a short time; Agile soon started to proliferate like fire, but something was off; it came in the form of Scrum. Now, 20 years later, Scrum is almost synonymous with Agile, but not that Agile from the Manifest, a new form of Agile hijacked by corporate that barely has any semblance with the old one.

To be able to sell the Agile idea to businessmen, they had to adapt it until the point it was not what it was proposed anymore. In order to keep managers happy, metrics and processes were introduced so they could regain their false sense of control. Processes and tools slowly replaced individuals and interactions. More and more, Scrum was becoming a monster of meetings and documents.

So instead of cultivating a team where everyone feels free to speak at any time about anything, valueing individuals. We now have Sprint Retrospective, Team Health Check, and many other useless meetings with predefined processes, going exactly against the Agile Manifesto.

I won’t even talk about the abomination of Scrum Poker; I’m sure some heartless sadist created it. But wait, with all these, the managers can print nice graphs of velocity or some other useless graph that give them this false feeling of control, which again goes against the Agile Manifesto, but who cares about it at this point, right?

To be honest, I met so many “Scrum Masters” who didn’t even know the existence of the Agile Manifesto, as if it is erased from software development history.

But don’t take my word for it; listen to what people like Allen Holub and Uncle Bob have to say.

All in all, we are moving in the wrong direction, we let business people decide how software engineering should work for so long that now what we call Agile is not much different than the RUP, and we are all witnesses of the slow death of Agile, killed by Scrum.

Google FLoC {Insert some F*K pun here}

2021-05-04T00:00:00+00:00

I’ll have to put a disclaimer on the beginning of this post: I don’t like Google, its practices, its mindset, their business model and everything in between. I have degoogled myself as much I could, and I keep looking for better alternatives in the sense of usability, privacy and security. However, on this post I’d like to go beyond the buzz created by their FLOC ID recently announced and already under attack for all the sides, based on its reception (Brave/Vivaldi/Firefox, all already said they won’t implement it) it looks like it is an dead-on-arrival technology. Does it really deserve all the hate?

FLoC(Federated Learning of Cohorts) is part of the Privacy Sandbox Proposals aiming to satisfy advertisers needs without third-party cookies and other tracking mechanisms. The idea behind FLoC is to cluster large groups of people to make them unidentifiable as individuals, but still relevant to ads in the form of a generic profile that they call FLoC cohort. In this sense, FLoC is more of a profiling tool than a tracking one.

Is it even possible? After the announcement a lot of criticism has been thrown on the proposal, some of them are valid points, some of them seem to not understand how it works.

EFF was one of the first ones to attack the proposal, they raise a valid point; Google is saying that a cohort ID can’t fingerprint a user due to how generic it is (on its first implementation it uses a 8-bit identification, which makes it only 256 possible cohorts), however, in combination with other fingerprinting techniques it can helps to identify a user even easier than now. Google, on the broader Privacy Sandbox, has other proposals to mitigate fingerprinting:

Creating Privacy Budget limits, where the website has a limited access to information that could be used for fingerprinting.
The User-Agent won’t give the full information that is shared Today, instead they will expose an API, which will be limited by the Privacy Budget above.
IP Security will make the IP available on request and part of the Privacy Budget

However, as pointed by EFF, none of these are currently implemented, and FLoC is already on Original trial, so for now we don’t have the other benefits, only the cons of the FLoC being used in conjunction with other techniques for fingerprinting.

The cohort is based on your web history, renewed weekly, and supposed hiding sensitive information to avoid homogeneity attacks, websites categorized as “sensitive data”, like medical or adult content won’t be showed there(the sensitive categories are the same already defined by Google for advertisement), this is achieved by ensuring that no cohort has more users who has visited sensitive websites more often than the general population, this approach is explained in more details in this paper.

They also will be auditing running frequently to identify cohorts that could make correlations leading to discrimination, again, not clear how efficient this can happen. FLoC delegates most of computing to the browser, keeping it locally, which will be responsible to calculate your cohort applying a SIMHash algorithm and expose it to the website via a browser API as below:

cohort = await document.interestCohort();
url = new URL("https://ads.example/getCreative");
url.searchParams.append("cohort", cohort);
creative = await fetch(url);

Another criticism is that as the cohort changes overtime, a website could use to collect more and more information of the user, creating a much broader profile. Google has a proposal for tackling this on their Longitudinal Privacy section: Sticky the same exposed cohort for each website you visit. Therefore, the first time you visit a website it will gain access to your cohort, which will remain the same on next visits, even if your local one has changed. However, this is just a proposal and raises other concerns per se.

Also, using a Default Allow is enough to raise some eyebrows. There is also some concerns about Google enrolling users on the trial without consent, both from user’s and from the websites.

In summary, I think there are some good ideas, ideas that may work better in terms of privacy than the current third-parties cookies approach (not a high bar, I know). However, for it to make sense, not only the FLoC from the Privacy Sandbox needs to be implemented, but most, if not all, of its proposals. Adding only FLoC to the mix is not replacing third-party cookies or fingerprinting, it is just adding another layer of data collection while preserving the web status quo, not the user’s privacy. I don’t think FLoC is where we should start this. Maybe starting from the Privacy Budgets Limits and Trust Token API will have a better reception?

When Horizontal Auto Scaler is not fast enough

2021-04-13T00:00:00+00:00

Recently I was listening to a podcast with the brazilian team responsible for probably the most popular fantasy game in Brazil: Cartola FC. Cartola FC is a fantasy game based on real football matches, and in a country that loves football as much as Brazil you can imagine the load they receive when a match is ongoing. The number of requests on their application has a very high variability, depending on whether there is a football match happening or not and whether some popular players have scored.

During these moments when a popular player has scored, the users wanna quickly check their status on the game, and then their API access can go, in a couple seconds, from ~30k requests por second to ~230k. Of course, when it happens they want to be able to have enough resources to not frustrate the user (he will be frustrated enough watching his selected players missing the goal).

There are already one big factor that makes this team “unique”, their infrastructure is on-premises. In a world where most of the computing power is centred on Amazon, Google and Microsoft it is sure a risk decision per se. They are asked about how they scale their pods when those spikes happen, and they comment that they don’t use things like Kubernetes Horizontal Auto Scaler, and the reason is quite simple, with the spike happening in a couple seconds the auto scaler is not fast enough to provision the extra pods.

Reading online about the Kubernetes HPA most of time its limitations are not mentioned and it sounds more like magic than actual technology.

At my current company we have a similar issue, as a tv provider, when a new football championship or Formule 1 season is about to start our APIs experience a ten fold increase in traffic, and even with HPA in place, sometimes they are just not fast enough.

While a pod is being prepared to become ready for new requests, all the load is put on the back of the current pods, eventually they can’t handle the requests anyre and then they become unhealthy, so Kubernetes takes them out of the pool and the problem only gets worse.

There is also other limitations like costs, nodes available and database connections, one failure can propagate the error down the hill becoming harder to recovery.

It is an ongoing and learning process, and we are playing with different scenarios and solutions in order to optimize the resources usage.

A few things that can help to mitigate the issue:

Scheduled provisioning

When an known event that increases traffic is about to happen you can provision before hand the necessary pods, increase the number of minimum pods, this is something that gets better as more data of past events you have. You also can use tools for load testing, i.e: Locust to simulate the number of requests you expect. Unfortunately, not always it is possible to predict when it will happen.

Higher liveness probe

This way your pod is not killed while it waits for the new pods to enter the pool, but make sure that you don’t let it too long to have stalled pods.

Increase sensibility of the HPA trigger metrics

Another approach you can try is to make the pods more sensible to trigger the scale more often and give more time for the current pods to handle the requests.

In summary, sometimes, scaling your application in a proper way is more of an art than a science, and you have to weight all the trade-offs in each possible solution, mix them, test and analyze to find the optimum way according to your situation, and then keep re-evaluating it and improving it, as new solutions and tools never stop coming.

Kubernetes HPA is an extraordinary tool, but it needs to be separated from the hype, like mostly things in this industry, but this a topic for another post.

Browser Isolation

2021-04-08T00:00:00+00:00

CloudFlare has just announced what they call zero-trusted browsing.

The services, aiming on business, has the goal of making the employee experience feel less restrictive in comparison to regular organization policies, while keep the browsing safe. The way it works is different than regular remote browsing, and they claim it is what it makes it faster and a frictionless experience.

If they delivery what they are promising it might set another level on business security without compromising the employee experience, as browsers has become the main attack vectors, but it all comes with a cost, security is not free after all.

This technology came from their acquisition of a company called S2 Systems, it’s a proprietary patent that attempts to tackle the main issues of the current remote browser isolation technologies.

The current technologies are “Pixel Pushing” and “DOM Reconstruction”.

Pixel Pushing: The rendered page is transmitted pixel by pixel, similar to remote desktops, this approach tend to be very slow and produces a terrible user experience.
DOM Reconstruction: The server “cleans” the DOM and send it back to the client, this approach is not as slow as the first one, but can break pages or even let malicious code pass undetected.

S2 System introduced what they call “Network Vector Rendering” or NVR, this technology utilizes Chromium Skia 2D library to send compressed Skia draw commands to any HTML5 compliant browser. In order to run it the local browser will have a NVR Web Assembly library responsible for draw the commands on the local window, and on the cloud it runs a headless Chromium-based browser.

I’m curious to try this approach, My only experience with browser isolation was running Citrix and its awful usability. Hopefully, CloudFlare will set the bar higher for this one, and make organizations a bit more… secure.

If you want to read more about CloudFlare browser isolation, check their official page

Hello, World

2021-04-07T00:00:00+00:00

Opening the season for posts with our traditional hello world!

You can expect posts about cybersecurity, privacy related news, scalability and all around programming.

So, let it begin!