BC TECH
Saved by @Jacobtldr

We just launched a fun little tool called Phantom Analyzer. It’s a 100% serverless tool that scans websites for hidden tracking pixels.

I want to talk about how we built it 👇

 The idea came about early this year after @mijustin gave us an idea about badges/certification, which developed into “what if we could scan websites for Google Analytics?!”. But we left the idea in Basecamp for many months.
Fast forward to Halloween, we’re thinking about fun ideas we could do to entertain people. We discussed “Phantom Analytics” in response to people getting confused with our product name, and had various ideas. And then we landed on the URL analyzer idea.
Once we’d finalized the spec, Paul got to work on our Halloween themed product and then coded up the HTML/CSS for it all. I then took it, put it into a Laravel application. Nice and easy.
Off the bat, I already knew the base stack I was using.

> Laravel Vapor
> ChipperCI for deployment
> SQS for queues
> DynamoDB for the database

We went with DynamoDB as we don’t want to worry about our database scaling!
So with our infrastructure known, we had a few challenges left to solve:

> How will we scan websites for tracking pixels?
> How will we utilize the queue and check the job is done?
> How will we validate the URL?
For scanning websites, the first thing I did was write out a complex, multi-level regex matching, guzzle executing scanner. But the problem was that it didn’t automatically run the javascript, which often includes additional requests, so the results weren’t accurate.
After spending a long time on that, I was searching for website crawlers and came across Puppeteer, which is a headless Chrome Node.js API. I then searched for how to get it running on Laravel Vapor and saw that someone had already solved that challenge!
I then spent 8 hours trying to start from scratch with Puppeteer, copying from @spatie_be’s Browsershot code, but I just couldn’t get it working. I went to bed, woke up the next morning, and decided that I’d start with Browsershot and simply modify it to what I needed.
I woke up the next day, and within 15 minutes I get a screenshot generated on Laravel Vapor, hooray! I then start to modify Browsershot…

Wait a minute...

Yes, out of the box, Browsershot already had what I needed. Are you kidding me?
So I modified my job and had it all working within minutes. Browsershot returns a list of network requests when loading a web page and returns them, bloody perfect. I then simply compared them against a list of around 10,000 known third party trackers that we had.
The next step was working out how we would get permanent storage in DynamoDB without bringing in anything extra. I wanted to keep it simple. So with DynamoDB as the driver, and “resources” as the storage root, I wrote a command that cached the tracking pixels indefinitely.
One of the initial concerns I had was regarding security. We would be passing user input to the command line and that wouldn't be safe. I spoke with @marcelpociot and he gave me some great advice, and I added in some validation. The active_url rule is fantastic.
I also wanted to have a way so that if a user entered a full URL (e.g. https://t.co/66d4eLDaOu) and not just "https://t.co/GA31muKcta", it still redirected them to the correct results page. Especially since our "tidying up" was opinionated. So we ran this code.
We then had to think about how to configure our Vapor app, and I went with the following settings

> 1024MB of RAM
> 2048 of RAM for the queue (could likely reduce!)
> Warm of 500
> CLI Timeout of 180 seconds

Those settings all worked nicely.
For our Vapor layers, we ran it like this. Very cool. My first time using Layers. Incredible work by the Vapor team (@themsaid @taylorotwell @enunomaduro).
For the "is it ready?" check, I debated using a UUID but I decided that we might have multiple users trying a website at the same time, and they should benefit from the same cache entry (we cache results for 5 minutes).
So for the ping, we went super old school. Interval and redirect when done. Very effective. And when it reloaded the page, it would hit the cache, see the entry and display it.
All in all, this was a fun project to build. I love working with Paul. The only design addition I made was the bats & fade, Paul did everything else. Very grateful for that 😂
I am still in awe over how quickly we deployed this with Vapor. I'm not kidding, it was all coded up and we just created it in the UI, deployed it and we were done. Remarkable experience. Infinite scale without any server work 😎

Hope you all enjoy Phantom Analyzer!

More from Tech

Simon DeDeo
Simon DeDeo
@SimonDeDeo
So, today, for the first time in 25 (!) years of Apple, I downgraded. From the 2016 MacBook Pro to my 2013, which I had kept in a drawer...


It was pretty simple to do—Apple Time Machine backups let me do it with one click.

That first tweet captures, in two pictures, how badly Apple has “lost the plot” (to quote @wylieprof). On the right is the Apple MagSafe adapter, from 2013. On the left, what I had “upgraded” to.

Thanks, Apple! I really was nostalgic for worrying about yanking my computer off the table.

Oh and I really appreciated not knowing if my computer was charging. What was great was the little whoop sound you used, so that the speaker before me could be informed I was charging my laptop.
TECH
Devdutta
Devdutta
@Devdutta96
Please please please take your cyber security seriously even if you don’t consider yourself a high risk person. Here are a few easy things to start with. 1/n

Switch from WhatsApp to Signal and enable disappearing messages. Most of us cannot stop using WhatsApp entirely but at least turn off back ups to Google Drive or iCloud. 2/n

Do not have private conversations using the messaging service on any social media platform like Facebook, Instagram or Twitter. 3/n

Switch from Gmail to ProtonMail or any other secure email service provider. 4/n


Switch from Google Docs to CryptPad or RiseUp Pad.
CYBER SECURITY , IT PRIVACY , CYBER SECURITY STEPS
Advertisement
JBD
JBD
@rakyll
Google's interview system is not great but there are various reasons why it feels less than it is. Most candidates don't know what the goal of our interviews are. First rule, it is not about finding an optimal solution or any solution at all.

You are interviewed for multiple skills simultaneously. Cognitive skills, communication, leadership are a few to name. If the point is not finding a solution, then what is it? Let me explain.

Your interviewers try to understand what it feels like to work with you on a daily basis. An interview question is just a tool in achieving that, it is not there to specifically measure your skills on a topic but a tool to understand the depth of your thinking.

Before the interview starts, ask them what they want to get out of this interview. Good interviewers should already have a plan and a set of expectations. Ask them what you should do. Don't start coding yet. Ask them you should produce. Discussion, diagrams, pseudo code, code?

Then, start cracking the question. List whatever questions you think it is important to solve this question, ask your edge cases. Get to a point where you are discussing about pros/cons of the solutions. These steps are critical. Don't just start coding. Have a consensus first.
TECH
Pete Savin
Pete Savin
@pete_savin
Wellcome to our virtual walk from Ambleside to Howtown on Lake Ullswater. I’ve based the route on #HadriansHighWay by @fellranger1 which he uses a variant to the main route. Books can be bought direct or through @VindolandaTrust


A view of #Ambleside from Wansfell our starting point in the heart of the #LakeDistrict @CumbriaMagazine


We can start at the roman fort at the edge of #Windermere. The Romans stated to push into the lakes in the early 90’s AD, this fort dates to the rebuild in Hadrian’s Reign possibly after warfare which brought about #HadriansWall #Ambleside


The #Ambleside posting wasn’t always a peaceful one as the gravestone in the @ArmittMuseum shows one man was killed “in the fort”


Time to leave and head out of town passing this scale model of the famous Bridge House in #Ambleside #virtualwalk
TECH
Knarf
Knarf
@Knarf93
@zkat__ I'm in the position that I actually find npm / yarn the best ecosystem. Whenever I use something else I always end up stubbing my toe into something thats missing / feels wrong.

Ex. Cargo seems to neither have a concept of devDependencies nor peerDependencies.

I also can't understand why it wouldn't have an "add" command to add a new dependency. And I'm no fan of Toml, json is greate (easy to parse and build tooling around), and the better option in my opinion would be json5.

C / C++ seems to just not have language package managers. The linux / bsd crowd seem to have decided that the system package manager also should be the language package manager. Which might have been fine if every Linux distro used the same system package manager.

Instead we end up with a N x M problem. Where we have a bunch of different operating systems and they all support multiple system package managers. So there's no easy way of distributing, referencing and updating C / C++ packages.

It is also my opinion that the compiler / runtime should be a package dependency. I don't like Rust's split between rustup and cargo (they should have been one tool). Similarly it would be better if you added Node as a dependency to package.json, that way we wouldn't need NVM.
TECH

You May Also Like

Anastasis
Anastasis
@OrdersofM
https://t.co/6cRR2B3jBE
Viruses and other pathogens are often studied as stand-alone entities, despite that, in nature, they mostly live in multispecies associations called biofilms—both externally and within the host.

https://t.co/FBfXhUrH5d


Microorganisms in biofilms are enclosed by an extracellular matrix that confers protection and improves survival. Previous studies have shown that viruses can secondarily colonize preexisting biofilms, and viral biofilms have also been described.


...we raise the perspective that CoVs can persistently infect bats due to their association with biofilm structures. This phenomenon potentially provides an optimal environment for nonpathogenic & well-adapted viruses to interact with the host, as well as for viral recombination.


Biofilms can also enhance virion viability in extracellular environments, such as on fomites and in aquatic sediments, allowing viral persistence and dissemination.
ALL
Manas Arora
Manas Arora
@iManasArora
My course is now available for FREE on youtube.

Channel link: https://t.co/BJqESqFVko
ALL
Advertisement
Delilah S. Dawson
Delilah S. Dawson
@DelilahSDawson
Hello, writers! How about a thread on LITERARY AGENTS? How to get one, how to talk to one, how to keep one happy. Spoiler alert: I am 100% pro agent and can't imagine not having one on my side. 1/

Why would you want a literary agent?
* you want to be traditionally published
* you want someone experienced to help guide your career
* you want to learn how to edit like a pro
* you want to sell foreign and movie rights
* you want answers to your newbie questions 2/

Why you might *NOT* want a literary agent:
* you want to self publish
* you're not willing to compromise on your edits
* you don't think their expertise is worth 15% of your advance
I... can't think this way. Literary agents have been crucial to my career. 3/

So, how do you get a literary agent?
1. Have a finished, revised, edited, polished manuscript.
2. Write a query letter for your book
3. Send your query to agents who rep your genre and are open to submissions
4. Repeat steps 1-4 until you're offered representation. 4/

So, let's go through those four steps. First of all, you must have a finished, revised, edited, polished book, and it must be sellable. That is, you can't sell a 600k picture book or a 40k adult Fantasy, etc. You must read extensively in the genre you're writing. 5/
WRITING
\u092c\u094d\u0930\u093e\u0939\u094d\u092e\u0923\u094b\u0902 \u0915\u093e \u0907\u0924\u093f\u0939\u093e\u0938\U0001f441\ufe0f\U0001f441\ufe0f
ब्राह्मणों का इतिहास👁️...
@BRAHMIN_0052
MOHIYAL_BRAHMIN_KING
RAJA_PORUS
King_Porus_History

1)King Porus was a famous Indian king, his real name was #Purushotama. He belonged to PURU tribe mentioned in #RIG_VEDA, the Ancient books of India.


2) Porus ruled the Punjab region and expanded his kingdom between the Jhelum River and Chenab River in the Indian Subcontinent. He was a legendary warrior and more war skills. He made the army most powerful with war skills. Even Alexander had also made difficulty to defeat Porus.

3)Word of Mouth Oral tradition in #Mohyal_Brahmin families since ancient times has kept the Great Saga of their Great Elder known to them as Maharaja “ #Puru” alive.

4) When #Alexander the Great invaded India in 326 BC he was challenged near Taxila by a mere chieftain of the Area. He was King #Porus, a #Vaid_Mohyal. After his encounter with Porus, Alexander met with more resistance near Multan where the forces were predominantly #Mohyal.

5)The Bali rulers of Multan fiercely engaged with Alexander's army in skirmishes in which many senior Generals died. Later during the long and bitter fighting in #Multan, Alexander got hurt by an arrow which pierced his chest ,
RELIGION
Vibhu Vashisth
Vibhu Vashisth
@VIBHU_Tweet
AGAMA SHASTRA:THE SCIENCE BEHIND THE MARVELS OF HINDU ARCHITECTURE

Agama Shastra is a Sanskrit word that means the manual for worship, temple building, rituals etc in Hindu Tradition.From Sanskrit,agama means handed down by tradition and Shastra means a commentary or treatise.


Therefore, Agama Shastra is a collection of ideas that drafts the rules for worship, temple construction, spirituality and rituals. It has been a guideline for many Hindus from ancient times. It's a collection of Sanskrit, Tamil and Grantha scriptures that mostly contains...


...methods of temple construction, idol creation, philosophical doctrines and meditative practices. It came into being after years of assimiliation and from a variety of sources. Some parts of it are pre-vedic while some are post-vedic.


Hindu Temples are not just temples, they are the pride and identity of Hindus. They are the symbols of Hindu unity and their religious aspirations. Temples are the places of spiritual healing and are our connection with the creator almighty.


Temples are the priceless treasure of our History, Heritage ,Art & Architecture. Architecture that mesmerises and spell bounds you. And Agama Shastra is that very text which is responsible for such a mesmerising and spell binding quality and beauty of our Hindu Temples.
ALL