BC SOCIETY
Saved by @jacobhtml

0/ When large eng orgs rely on metrics for both monitoring *and* observability, they struggle with cardinality.

This is a thread about “the two drivers of cardinality.” And which one of those we should kill. :)

🧵👇

 1/ Okay, first off: “what is cardinality, anyway?” And why is it such a big deal for metrics?

“Cardinality” is a mathematical term: it’s *the number of elements in a set*... boring! So why tf does anybody care??

Well, because people think they need it, then suddenly, "$$$$$$$."
2/ When a developer inserts a (custom) metric to their code, they might think they’re just adding, well, *one metric*. …
3/ … But when they add “tags” to that metric – like , , or (shiver) – they are actually creating a *set* of metric time series, with the *cardinality* of that set being the total number of unique combinations of those tags.
4/ The problem is that some of those tags have many distinct values.

E.g., when I was working on Monarch at Google, there was a single gmail metric that became over 300,000,000 (!!!) distinct time series. In a TSDB, that cardinality is the unit of cost.

So again, “$$$$$$$.”
5/ Okay, so that’s why people care about metrics cardinality. Now, what are the two *drivers* of that cardinality?

A) Monitoring: more detailed *health measurements*
B) Observability: more detailed *explanations of changes*

Let’s take these one at a time…
6/ First, “More Detailed Health Measurements” (monitoring):

Consider an RPC metric that counts errors. You need to independently monitor the error rate for different methods. And so – voila – someone adds a “method” tag, and now there’s 10x the cardinality for that metric.
7/ … And also 10x the cost. But that’s justifiable, as there’s a business case for independently monitoring the reliability of distinct RPC methods.

Put another way, you might rightly have different error budgets for different RPC methods, so their statistics must be separable.
8/ Bottom line: When it comes to “measuring health,” we often *need* cardinality in order to hone in on the signals we actually care about. Increasing cardinality to *proactively* monitor the signals we care most about is usually a worthwhile tradeoff.
9/ Now what about using cardinality for “More Detailed Explanations of Changes?”

This is the real culprit! And, frankly, should be abolished. :) Metrics cardinality is the wrong way to do observability – to explain changes.

More on that…
10/ Say your monitoring tells you that there’s a problem with a critical symptom – e.g., you’re suddenly burning through an SLO error budget at an unsustainable clip.
11/ After a painful outage, say you realize a single customer DOSed your service. So someone adds a `customer` tag “for next time.”

But this is unsustainable: each incident reveals a new failure mode, devs keep adding tags, and before long your metrics bill is out of control.
12/ The problem, in a nutshell:

Distributed systems can fail for a staggeringly large number of reasons. *You can't use metrics cardinality to isolate each one.*
13/ How *should* we explain changes to production systems?

Understanding change is the central problem of observability. Effective workflows might *start* with metrics, but they must pivot towards a multi-telemetry, multi-service guided analysis.
14/ So, to sum up: spend your limited cardinality budget on *monitoring*, and then look for observability that (a) naturally explains changes and (b) relies on transactional data sources that do not penalize you for high/unbounded cardinality.
15/ PS: For more on how to distinguish monitoring and observability, see this thread: https://t.co/2RHs36Nknc

PPS: If you’d like to discuss/debate/request-more-detail any of the above, reply to this thread or DM me!

More from Society

John Hayward
John Hayward
@Doc_0
It's funny how so many people who claim to worship "democracy" also dismiss nationalism as selfish xenophobia. Democracy is absolutely meaningless without strong national sovereignty. Under globalism, citizens of any given state have little influence over how they are governed.

Nationalism isn't just mindless tribalism. People are correct to insist their government has a responsibility TO THEM that vastly outweighs any imagined responsibility to foreign governments they have absolutely no influence over, or foreign populations.

We've seen that point taken to absurd extremes by the Left during the Trump administration, with whack-job judges ruling that foreigners have rights that EXCEED those of American citizens, including absolute rights to enter the U.S. and influence its government.

And we're frequently told that some international elite "consensus" completely overrides our ability to make individual decisions - or even to petition our own government, or vote for representatives who disagree with whatever "the world" has supposedly decided.

All of that is the absolute negation of "democracy" in any form, including the dangerous mob rule and majoritarianism that the Left selectively advocates. (Notice they always have a list of things nobody gets to vote on anymore, no matter how strong the majority vote would be.)
SOCIETY
Mike Romano
Mike Romano
@MikeRomanoNTCA
The numbers on their face for the @FCC RDOF auction are exciting - lots of Gigabit winners. But it’s important to remember the job is just starting. These rural areas don’t magically get Gig today or tomorrow. 1/

The job is only done and the celebration can really only begin once we see that rural Americans are actually going to get the broadband they’ve been promised. 2/

And that’s the rub here - at least from preliminary indications, there are LOTS of question marks clearly hanging out there once you look past the gloss and proclamations. And since we won’t know for maybe 5 years if it worked, due diligence is essential now. 3/

A number of big winners have offered some service but never done Gig. This would be like giving hundreds of millions of $ to firms that have repaved residential streets to now design & build our interstate highway bridges. 4/

A number of big winners are using tech that has done Gig (maybe) only in tightly controlled applications that don’t look anything like rural America. This would be like giving interstate highway $ to firms who’ve built lots of scale models but never actually built a bridge. 5/
SOCIETY
Advertisement
Louisette Lanteigne \U0001f30e\u270c\ufe0f\u2696\ufe0f\u2665\ufe0f
Louisette Lanteigne 🌎✌...
@lulex
Count Giancarlo Cicogna was a member of the Ciga hotels clan. He was with Ghislaine Maxwell for 4 years. He was 'the man who moulded the Ghislaine we now see. He told her where to get her hair cut, and what to wear.'
https://t.co/XorCl9VKW6


There is a very Ghislaine like character named Madam Anais in the movie La Belle De Jour who runs a Brothel. The film was produced by Marina Cicogna.


Private View of 'Once Upon Time' an Exhibition of Photographs by Marina Cicogna Sponsored by Fiat at the Little Black Gallery,

My view Marina Cicogna looks like Dottie in the Disney Channel series WandaVision.


NY Times covered a story of Countess Marina Cicogna, a woman with a very intetesting life. I am sure she knows many secrets and that gives her power over people of influence.
SOCIETY
Marshall Steinbaum \U0001f525
Marshall Steinbaum 🔥...
@Econ_Marshall
My wonderful @jainfamilyinst colleagues @laura_bmw & @Edi_Nilaj have released their new report "Student Debt and Young America," which pretty comprehensively demolishes many of the myths about who does and doesn't have student debt.
https://t.co/y02KFosRQ3
Highlights to follow
/1

@laura_bmw @Edi_Nilaj First of all, look at this one chart that tells you why the student debt crisis is happening: rising debt; stagnant income, and the gap has opened widest for minority borrowers.
/2


You look at that chart and you see the total failure of the human capital model to explain labor market outcomes, insofar as it even is a falsifiable social science hypothesis.
/3

Second, while it's true that total loan balances are increasing in income, student debt has increased most over time among the lowest-income groups. This tells you that student debt is becoming a marker of the disadvantaged rather than the advantaged.
/4


Third, balances on outstanding loans are rising over time, not declining. As I said in my last report, the student debt crisis is a crisis of non-repayment. This is why it's foolish to think the student debt is ever going to be paid back, whether it's cancelled now or not.
/5
SOCIETY
John Hayward
John Hayward
@Doc_0
Every crisis becomes a religion if it lasts long enough.

One factor in that transformation is the Beautiful Theory phenomenon: the power elite insists its remedies are logical and politically correct so they MUST work, even if the actual evidence shows they obviously don't.
SOCIETY

You May Also Like

Vibhu Vashisth
Vibhu Vashisth
@VIBHU_Tweet
शमशान में जब महर्षि दधीचि के मांसपिंड का दाह संस्कार हो रहा था तो उनकी पत्नी अपने पति का वियोग सहन नहीं कर पायी और पास में ही स्थित विशाल पीपल वृक्ष के कोटर में अपने तीन वर्ष के बालक को रख के स्वयं चिता पे बैठ कर सती हो गयी ।इस प्रकार ऋषी दधीचि और उनकी पत्नी की मुक्ति हो गयी।


परन्तु पीपल के कोटर में रखा बालक भूख प्यास से तड़पने लगा। जब कुछ नहीं मिला तो वो कोटर में पड़े पीपल के गोदों (फल) को खाकर बड़ा होने लगा। कालान्तर में पीपल के फलों और पत्तों को खाकर बालक का जीवन किसी प्रकार सुरक्षित रहा।

एक दिन देवर्षि नारद वहां से गुजर रहे थे ।नारद ने पीपल के कोटर में बालक को देख कर उसका परिचय मांगा -
नारद बोले - बालक तुम कौन हो?
बालक - यही तो मैं भी जानना चहता हूँ ।
नारद - तुम्हारे जनक कौन हैं?
बालक - यही तो मैं भी जानना चाहता हूँ ।

तब नारद ने आँखें बन्द कर ध्यान लगाया ।


तत्पश्चात आश्चर्यचकित हो कर बालक को बताया कि 'हे बालक! तुम महान दानी महर्षि दधीचि के पुत्र हो । तुम्हारे पिता की अस्थियों का वज्रास्त्र बनाकर ही देवताओं ने असुरों पर विजय पायी थी।तुम्हारे पिता की मृत्यु मात्र 31 वर्ष की वय में ही हो गयी थी'।

बालक - मेरे पिता की अकाल मृत्यु का क्या कारण था?
नारद - तुम्हारे पिता पर शनिदेव की महादशा थी।
बालक - मेरे उपर आयी विपत्ति का कारण क्या था?
नारद - शनिदेव की महादशा।
इतना बताकर देवर्षि नारद ने पीपल के पत्तों और गोदों को खाकर बड़े हुए उस बालक का नाम पिप्पलाद रखा और उसे दीक्षित किया।
ALL
Patrick McKenzie
Patrick McKenzie
@patio11
It’s underremarked upon that stablecoins accomplish basically all of the original design goals for Bitcoin qua a transactional mechanism (instant free value transfer anywhere), but that nobody cares about this because nobody ever cared about that part of the pitch.

“Why do people care about stablecoins then?”

A mix of “they encourage dollar-denominated liquidity in the cryptocurrency ecosystem and discourage withdrawal of the same” and “they’re good for money laundering.”

“But they make value transfer between exchanges much faster!”

This was a solved problem in traditional finance, too, mostly through the extension of credit. (It doesn’t matter how long settlement takes if there is sufficient trust to enable credit.)

The Bitcoin ecosystem is *positively allergic* to credit, so you have to call it a coin for them to accept it. And after you call it a coin they ignore everything the world has learned about credit, like risk management.

“Stablecoins aren’t credit!”

They’re pretty much exactly credit? A tether is a zero-coupon Bitfinex bond with a non-functioning call option. I
TECH
Advertisement
Anshul Pandey
Anshul Pandey
@Anshulspiritual
THE IMPORTANCE OF DEVI BEEJ / BIJA MANTRA.

Once Suutji in his discourse with King Janmajeya repeated the words of Vyasji regarding the pronounciation of the word ऐ. Vyasji had heard the importance of this word in a meeting of all important saints at Naimisharanya.


There this story was discussed.

In that gathering of such great sages, Rishi Jamdagni asked a question to all the eminent luminaries. He wanted to know which diety is the most revered and gets satisfied easily with our prayers. So Lomash rishi replied that Devi

Bhagwati was one such diety. She will be satisfied and is ready to grant you a boon even if you just say the first letter of her beej mantra. In this context he narrated the story of Satyavrat. The story is as follows.

There lived a Brahmin named Devdutt in Kosal.

He was childless. He arranged for an elaborate yagya for a intelligent son to be born. He invited all the eminent rishi's to conduct this Yagya. Rishi Suhotra, Pael, Brahaspati Yagyavalka were assigned different tasks. Rishi Gobhil was asked to sing the Saamved.

He sang the verses very well, but he had to take respite in order to breathe between two verses. This angered devdutt. He spoke his thoughts to Gobhil rishi.

The rishi got very angry as devdutt did not realise the practicalities of singing. He cursed devdutt that
ALL
Laurie Voss
Laurie Voss
@seldo
The 2018 State of JavaScript survey is out. They got 20,000 responses and have some delicious, delicious data. I'mma thread in some highlights:

First up: JavaScript flavors. There's TypeScript at 47% adoption, a tad higher than npm's own survey results (which said 46%). npm's survey is coming up again soon and will ask about TypeScript in a lot more detail.


I really, *really* like SoJ's "would not use again" question, which lets people who've abandoned a tech self-identify. This is noticeable in the graph above with Flow users -- 41% of people who've used Flow say they wouldn't use it again.

Now JavaScript frameworks. It's been a full year since npm's survey, so these numbers differ from ours in the directions I expected:
React 65% (vs. 60%)
Vue 29% (vs. 24%)
Ember 5% (vs 4%, I was expecting a bigger rise)

But there's a shocker in here: Angular.


npm's survey had Angular at 40% last year and SoJ has it at either:
- 58% (if you include those who don't want to use it again)
- 24% (if you count only those who like it)
Since npm's question didn't ask if they intend to *continue* using it I think that might explain this.
TECH
Billy Bostickson \U0001f3f4\U0001f441&\U0001f441 \U0001f193
Billy Bostickson 🏴👁&👁 ...
@BillyBostickson
@EricTopol @NBA @StephenKissler @yhgrad B.1.1.7 reveals clearly that SARS-CoV-2 is reverting to its original pre-outbreak condition, i.e. adapted to transgenic hACE2 mice (either Baric's BALB/c ones or others used at WIV labs during chimeric bat coronavirus experiments aimed at developing a pan betacoronavirus vaccine)

@NBA @StephenKissler @yhgrad 1. From Day 1, SARS-COV-2 was very well adapted to humans .....and transgenic hACE2 Mice


@NBA @StephenKissler @yhgrad 2. High Probability of serial passaging in Transgenic Mice expressing hACE2 in genesis of SARS-COV-2


@NBA @StephenKissler @yhgrad B.1.1.7 has an unusually large number of genetic changes, ... found to date in mouse-adapted SARS-CoV2 and is also seen in ferret infections.
https://t.co/9Z4oJmkcKj


@NBA @StephenKissler @yhgrad We adapted a clinical isolate of SARS-CoV-2 by serial passaging in the ... Thus, this mouse-adapted strain and associated challenge model should be ... (B) SARS-CoV-2 genomic RNA loads in mouse lung homogenates at P0 to P6.
https://t.co/I90OOCJg7o
ALL