BC MACHINE LEARNING
Saved by @Mollyycolllinss

Okay, doing my first baby steps with r2frida (which combines the power of @radareorg and @fridadotre).

Gonna share my progress in this thread (live, so keep calm).

The goal: Runtime inspection of data sent out by TikTok !!before!! it gets encrypted

1/many

 First of all, we do not start from zero. I got some prior knowledge from past reversing attempts and want to share some important facts.

TikTok's (log data) encryption is accomplished by a native library. The Android Java code just serves as proxy function to the native function
The decompiled code for the respective native JNI function of an older TikTok version looks something like this, but in this example I use the most current TT version (no statical analysis done, yet)
In case you never reversed native libraries which were build to interface with Android Java layer via JNI, I highly suggest the entry level introduction on the topic by @maddiestone

https://t.co/T63vo3N4fw
Before we start, I want to pinpoint some important aspects (which are also covered by Maddie's videos).

1) Unlike raw C-functions, JNI functions like the one showcased above, receive pointers to complex Java objects .

F.e. a function receiving a String on the Java layer...
... would receive a pointer to a 'jstring' on the native layer (not a zero-terminated C-String).

In order to retrieve a C-String, to go on working with it in the native code, some translation functionality is required. This functionality is provided by the ...
JNI (Java Native Interface). The JNI environment is passed in to JNI functions as first parameter.

If you look at the example screenshot again, you see exactly this. Functions provided by the 'env' pointer are used to parse the Java function arguments (f.e. jByteArrays) ...
Once the raw data is converted to a more C-ish form, it gets passed to a inner function 'ss_encrypt' in my example. The inner function, in this case, is a pure C function and thus receives only C-style parameters (also no 'env' parameter, so it would not be able to access JNI)
A 2nd important aspect on JNI libraries, covered by @maddiestone

2) There are two ways to expose JNI methods from a native library:

a) export them with proper naming convention, so that JNI could recognize same on library load
b) use the JNI functionality 'registerNatives'...
... to register the JNI functions once the library gets loaded.

The second method of registering methods is wel suited for obfuscated code, as the methods neither have to follow naming convention, nor do they have to be exported.
As you might expect, TikTok uses the 'registerNative' approach. The screenshot below shows log output from a custom tool, which monitors JNI methods registered by instrumented Android apps (TikTok's encryption method in the example)
If you would decompile the Java part of the TikTok apk, the encryption functionality (on an older version) would look something like this:
The Java method 'm18421a' receives a Java 'byte[]' and a Java 'int' as parameters and returns a 'byte[]', again.

Internally, this data is forwarded to the native JNI method 'ttEncrypt'.
The important aspect about this, is that the native 'ttEncrypt' JNI method has to accept those exact parameter types and thus has to "register" with a proper method signature.

We already saw this signature in a previous screenshot
The last line from the screenshot above, shows 3 things the native code has to provide for each method, when calling register natives:

1) the call address of the native function implementation (0x7d70d1d5 in example)
2) The function name (ttEncrypt)
...
3) The method signature, which is '([BI)[B' in this case and translates to:

'(' start of parameters
'[B' byte[]
'I' int
')' end of parameters
'[B' byte[] (return value)
So we keep this in mind: Even if the native library does not export the encryption method, it has to store the 1) funtion address, 2) name and 3) signature in a data structure, in order to provide it to 'registerNatives' once the library gets loaded by JNI
We now know almost everything we need, in order to head over to r2frida, except some important facts on my test setup:

- the app is inspected on a physical device, running Android 9
- the device uses a !!32bit!! ARM application core
As I am new to r2frida, chances are high that things could be achieved in an easier ways.

Now to get started, I already have the latest @fridadotre server running on my USB connected android device and 'frida-ls-device' shows it being ready-for-action
So let's spawn a fresh TikTok instance with r2frida, using the following command
The command 'launch'es the process, on the attached frida 'usb' device with the device id '4c197256' and the process's package name, of course is 'com.zhiliaoapp.musically'

Instead of 'launch', two other options could be used:
- 'attach' (would attach to an already running process, given by name or PID)
- 'spawn' (like 'launch', but the process would not be resumed automatically after attaching)
So for the warm up, let us use the Frida functionality, which alllows enumerating loaded Java classes. This nicely combines with the r2 syntax (concatenation of single-letter commands, '~' for grep)

Important: commands targeting the r2frida plugin have to be prefixed with '\'
The r2frida command to list classes is '\ic' (note the backslash prefix). The unfiltered result would be a bit overwhelming ...
... so we grep for classes including the term "crypt"
The '\ic ' command lists the !Java! methods of the respective class
The signature of the static method 'EncryptorUtil.a' should look familiar to us (if you read the first tweets). It represents the Java layer of the encryption method and is called 'a' in this version
Note: The information above would be enough, to Intercept the method from the Java layer (f.e. with @fridadotre or Xposed), in order to inspect the call arguments (the byte[] parameter represents the plain data before encryption)
... but we are here for the native layer and to inspect data at runtime, right?

So lets search the whole address space for our native method name 'ttEncrypt'

Note: If you'd use r2's ascii search nothing would happen, you have to use the '\' prefix to search with r2frida
The search ends with two hits:
The screenshot below shows, that the attempt to print a hexdump from the address of the first hit fails with r2, while r2frida (backslash prefix) works.

Reason: The memory region was not populated when r2 was started (encryption library was loaded after process launch)
I solved this issue like this:
1) Quit r2
2) Open r2 with r2frida, again, but this time **attach** to the already running process

et voila ... the memory offset is mapped and dumpable with 'px' (without backslash prefix)
Note: The last step is not necessary for a data hexdump, as you could still use '\px', but it turned out to be useful when it comes to printing the disassembly of "late loaded" code regions. This is because I sometimes struggled with '\pd', but 'pd' worked (+ various r2 views)
Having a closer look at the first hit of out string search for 'ttEncrypt', we notice that it is directly followed by a C-string with our method signature.

So chances are high, that this data is part of the structure which gets handed in to 'registerNatives'
Reminder: in order to register the 'ttEncrypt' method to JNI, the 'registerNatives' method requires a structure containing
- method name (C-string)
- method signature (C-string)
- method pointer (native pointer)
So the next step would be to search the process memory space for cross references to the address of this method name string (0x8448b74c). As I haven't applied any auto analysis, I use a simple hex search for this (in my case the byte order of the address has to be reversed ...
... to account for the architecture endianess).

The result is promising: Only one hit, for a search across the whole address space:
Printing the first 12 bytes from this XREF offset, reveals 3 pointers again (reversed endianess):

- 0x8448b74c (expected, method name pointer)
- 0x8448b756 (ptr to signature string, yay)
- 0x8448b1d5 (likely pointer to JNI method implementation)
So the layout of the 3 pointers from above speaks a clear language. Very likely this is the data struct passed in to 'registerNatives' and thud 0x8448b1d5 points to the native implementation of 'ttEncrypt'
Sorry, before going on I have to insert a small excurse on adressing/instruction sets on arm 32 (specific to my test setup). Anyways it is crucial:

Arm 32 supports two instruction sets "ARM mode" (32bit) and "Thumb mode" (16bit) which could be used interchangebly
In order to distinguish if a function call target (branch) should be interpreted as ARM or THUMB, the least significant bit (LSB) of the function address is taken into account

For ARM the LSB is 0 (even address)
For THUMB the LSB is 1 (odd address)
The actual instruction ALWAYS resides on an odd address.

This means the function address 0x8448b1d5 homes code in THUMB mode (16bit), while the first instruction resides at 0x8448b1d4

(sorry if it gets a bit complicated, will be clear in a second)
So if we print the disassembly at the (assumed) 'ttEncrypt' address, things look a bit weird
... this is because instructions are interpreted in ARM mode (32bit).

Lets fix this:
... looks much better, still the first instruction is off-by-one 🤣

No seriously, as explained, on arm32 we have to disassemble at [THUMB mode address - 1] = 0x8448b1d4
Nice, this looks like a proper function stub (note how the callee stores reg values on the stack, before moving on).

Now to get a feeling on how often this function is called, lets use 'r2frida' power to trace it.

Important: The thumb address has to be used here!!!
The resulting output of the '\dt' command, which places the trace hook also indicates that the function address maps to an offset in 'https://t.co/fxhDnQke2o' ... let us call this a "nice confirmation"

Some actions in the TikTok app ... trace logs for ttEncrypt-calls arrive
... cigarette break ... stay tuned (if the app crashes meanwhile, I'll start from scratch)
Let's remove the trace hook for now, with '\dt-*'
Remember my screenshot of a decompiled 'ttEncrypt' function from an older TT version. We traced the corresponding functions.

Trying to runtime-parse the function parameters, which represent Java object instances would be insane (maybe impossible)
... luckily, at least the old implementation, internally called a method 'ss_encrypt' which received a c-style byte array pointer and an integer representing the length as first two parameters.
It would be way easier to runtime-inspect these
Lets take a closer look on the disassembly of our assumed 'ttEncrypt' function, by seeking to its offset with 's 0x8448b1d5' and switching to a more suitable r2 view with uppercase 'V' command (press 'p' till the view changes to disassembly)
The view from above allows scrolling through the functions code with cursor keys. Most important: calls to other code parts (branches) are printed bold and suffixed with [1], [2] ...

Hitting [alt+1] moves us straight to the marked branch offset:
The code above looks not like a legit inner function (we do not care for alignment and inspect the next branch).

Hitting 'u' returns us to the parent function, followed by [alt+2] which brings us into the 2nd branch
The 2nd branch at 0x84483aa4 looks better (proper function stub). We could easily drift back to the static analysis world, to find further evidence for it being the inner 'ss_encrypt' function. But hey, we are working with instrumentation, so let us just inspect the calls
Remember: While we disassembled at 0x84483aa4, the code is THUMB mode. Thus the proper tracing address would be 0x84483aa5 (LSB set to 1), unlike you like crashes (restarting here would not be funny, 'cause thanks to ASLR all function offsets would differ)
In contrast to our first tracing attempt, we use the beautiful command for formatted tracing, which allows us to print out function parameters for each call in a predefined format.

Command syntax:
The screenshot below shows how I placed my trace hook. The 'pppp' means that the first 4 function parameters should be printed as hex values (pointers) for each call.

Ultimately 2 calls get logged

More from Machine learning

Mads Brodt
Mads Brodt
@madsbrodt
I've applied for a total of 10 development jobs in my life:

- 7 lead to interviews
- Of those 7, 4 was a good fit for me
- Of those 4, 3 offered me the job
- In all 3 cases, I negotiated a better deal

🧵 on what worked for me and what YOU can do to get similar results 👇

1. Apply for the RIGHT jobs. Look for midsize to large companies, not necessarily tech related. These tend to get way fewer applications than traditional tech companies, so it's easier to get your foot in the door - and they also tend to be great environments for learning.

2. Spend time on your application. Make sure your resumé and written application is tailored to the exact position. Recruiters or hiring managers can smell a copy/pasted resumé from miles away. Use as many words from the job spec in your application as possible.

3. Have a portfolio. Show off your skills and some personality on your website. This is a nobrainer for front-end devs, but it's true for other areas too. It's a great way to stand out over other candidates by actually *showing* the value you can create and your experience.

4. Establish personal contact. Send a quick message or make a phone call before applying. Just introduce yourself to the person in charge of hiring and ask a simple question. It's the best way I find to make them remember your application when you submit it.
MACHINE LEARNING
Unknown \U0001f464
Unknown 👤
@console_404
10 PYTHON 🐍 libraries for machine learning.

Retweets are appreciated.
[ Thread ]


1. NumPy (Numerical Python)

- The most powerful feature of NumPy is the n-dimensional array.

- It contains basic linear algebra functions, Fourier transforms, and tools for integration with other low-level languages.

Ref: https://t.co/XY13ILXwSN


2. SciPy (Scientific Python)

- SciPy is built on NumPy.

- It is one of the most useful libraries for a variety of high-level science and engineering modules like discrete Fourier transform, Linear Algebra, Optimization, and Sparse matrices.

Ref: https://t.co/ALTFqM2VUo


3. Matplotlib

- Matplotlib is a comprehensive library for creating static, animated, and interactive visualizations in Python.

- You can also use Latex commands to add math to your plot.

- Matplotlib makes hard things possible.

Ref: https://t.co/zodOo2WzGx


4. Pandas

- Pandas is for structured data operations and manipulations.

- It is extensively used for data munging and preparation.

- Pandas were added relatively recently to Python and have been instrumental in boosting Python’s usage.

Ref: https://t.co/IFzikVHht4
MACHINE LEARNING
Advertisement
emmi
emmi
@emmibevensee
The "polarization is the problem" narrative to me is bad because it implies the status quo was a stable and acceptable set of trade-offs. It's the equivalent of a pollyanna "why can't we all get along" but the we in question is white supremacy and its targets.

like i understand that conflict mediation requires mutual concessions (i did my MA in conflict studies) and while it's important to meet people's deep needs across spectrums of difference, certain things are just not up for negotiation.

Also while peacebuilding work is good. It's not just "seeing each other as human" it's literally about transforming structural issues. (hence Galtung's distinction between positive and negative peace wherin negative peace is like a ceasefire and positive is a transformed society.

And I think that to whatever extent that kind of peacebuilding work is applicable, it's not between fascists and people who think fascists are bad. Fascists are not invited. They are only invited if they seize territories and can't be crushed. Then secret negotiations.

The US is always like "we don't negotiate with terrorists" but literally every president negotiates with who they consider to be terrorists. It's all just down low. But that's just because counter-insurgency failed.
MACHINE LEARNING
Santiago
Santiago
@svpino
An introduction to one of the the most basic structures used in machine learning: a tensor.

🧵👇

Tensors are the data structure used by machine learning systems, and getting to know them is an essential skill you should build early on.

A tensor is a container for numerical data. It is the way we store the information that we'll use within our system.

(2 / 16)

Three primary attributes define a tensor:

▫️ Its rank
▫️ Its shape
▫️ Its data type

(3 / 16)

The rank of a tensor refers to the tensor's number of axes.

Examples:

▫️ The rank of a matrix is 2 because it has two axes.
▫️ The rank of a vector is 1 because it has a single axis.

(4 / 16)

The shape of a tensor describes the number of dimensions along each axis.

Example:

▫️ A square matrix may have (3, 3) dimensions.
▫️ A tensor of rank 3 may have (2, 5, 7) dimensions.

(5 / 16)
MACHINE LEARNING , TENSOR BASICS
Santiago
Santiago
@svpino
11 key concepts of Machine Learning.

— Supervised Learning Edition —

🧵👇

😜

Before starting, remember that, if you follow me, one of your enemies will be immediately destroyed (and you'll get to read more of these threads, of course.)

And if you don't follow me, well, you just hurt my feelings.

😜

1. Labels

(Also referred to as "y")

The label is the piece of information that we are predicting.

For example:

- the animal that's shown in a picture
- the price of a house
- whether a message is spam or not

👇

2. Features

(Also referred to as "x")

These are the input variables to our problem. We use these features to predict the "label."

For example:

- pixels of a picture
- number of bedrooms of a house
- square footage of a house

👇

3. Samples

(This is also known as "examples.")

A sample is a particular instance of data (features or "x.") It could be "labeled" or "unlabeled."

👇
MACHINE LEARNING

You May Also Like

Mau Abata \U0001314a
Mau Abata 𓅊
@MauAbata
Read through a forum thread on Chastity Mansion to see who is talking about the Edge-o-Matic and figure out what more to address regarding frequent questions, as well as scout out user feedback. Here's some thoughts you can carry back to the forums!

My website doesn't have any product reviews, but people scatter them everywhere else in forums, tweets, xtube vids, etc.

If you ARE a previous customer, and you like the thing, can you drop a review on my site to help others?

The price is steep, but that's the amount of coins I calculated to make these. I have several spreadsheets calculating every expense involved with this project.

The current cost sustains production and development of new tech, and is helping create jobs. We are 2 people now!!

It IS 3D printed, because injection molding is orders of magnitude more expensive for the initial tooling, and I am still making small revisions to the enclosures.

Injection molded plastics are a six thousand dollar commitment for fifteen cent parts. Great for mass mfg.

Also, if you didn't know, 3D printing technology allows for some shapes which would be difficult, expensive, or impossible on injection molding or milled parts!!

Undercuts, which I use in this design, require special lifters to mold that cost a tremendous fortune.
FOR LATER READ
Sara Benincasa
Sara Benincasa
@SaraJBenincasa
Spent a quaint half hour in a cafe listening to a guy yell at his dad after his dad said that the women Trump molested wanted it: "Dad, that's like saying the Jews who died in the Holocaust were asking for it!" ENJOY YOUR TERRIBLE FAMILIES THIS WEEK!

"Mom, has a man ever touched you without your consent?"
"Of course."
"Did you report it?"
"No."
"Dad, your WIFE has been through this!"

Dad was unmoved and told the guy to stop arguing with him.

At this point we learned the boy child has allergies.

I forgot to remark upon the presence of the very cute grandson, who was very small and gave not one shit about the yelling. I believe a sister or aunt was present and feeling very stressed. I wanted to hand her a Xanax or seven.

It was really hard to not jump in and say, "Sounds like you guys are going to have a fun meal tomorrow!" so they could unite in hatred against me, the outsider, and stop squabbling. Instead I exchanged glances with a nearby fellow lady patron, who also emotionally ate a cookie.

Anyway, I want to thank the random family for yelling about rape and the Holocaust in an Israeli cafe in the US of A today because it reminded me that while this country isn't the greatest, it is inevitably my stupid favorite and I am never, ever bored.
SOCIETY
Advertisement
Bharadwaj
Bharadwaj
@BharadwajSpeaks
Today is Jayanti of Ahilyabai Holkar

She was a widow who became Malwa's greatest queen. Defended our country & personally led armies into battle.Never plundered anyone. Developed Malwa into a prosperous kingdom. Rebuilt temples destroyed by Mughals. Built Dharmashalas at Tirthas


Unfortunately, her memory has been blemished by inaccurate representations in modern popular media.

The serial "Punyashlok Ahilyabai" depicts her as ultra secularist feminazi icon who questions Hindu rituals & Pandits while patronising Sufis.

This is far from historical truth!


Ahilyabai was born on May 31 1725 to Patil Mankoji Shinde.

She was from Dhangar clan(classified as OBC today). While Dhangars were traditionally into herding, they took up arms & held several military /civil roles in various eras of history(especially during Maratha resurgence)


In 1737, when she was 12, Ahilyabai Holkar was married to Khanderao , son of Subedar Malhar Rao Holkar.

By the time she was married, fortunes of Holkar family had already waxed.

Malhar Rao was a Subedar of territory comprising 30 Parganas yielding a revenue of at least 7 Lakh


In 1741, when Ahilyabai was newly married, her father -in-law Malhar Rao Built Indore palace & ordered the kanungo to populate the settlement with a promise of protection to traders.

This began rise of Indore from its background as modest trading town Indrapuri to a great city
ALL
WAKE UP AUSTRALIA WAKE UP
WAKE UP AUSTRALIA WAKE...
@WakeAustralia
Facebook originally a CIA program called "LifeLog".
LifeLog, via DARPA, terminated on Feb 4th, 2004.
Facebook was launched on Feb 4th, 2004.
Many of the LifeLog team became execs at FB.
Zuckerberg is a figurehead.
CIA allowed Cambridge to help Trump win

https://t.co/enzOXDCogV


Pentagon Kills LifeLog
BUSINESS
(\u3063\u25d4\u25e1\u25d4)\u3063 \U0001f499 \U0001d440\U0001d4b6rymo Belfast \U0001f499\U0001f331\U0001f340 \u24cb\U0001f397\ufe0f
(っ◔◡◔)っ 💙 𝑀𝒶rymo Belfa...
@MarymoBelfast
THREAD PART 1.

On Sunday 21st June, 14 year old Noah Donohoe left his home to meet his friends at Cave Hill Belfast to study for school. #RememberMyNoah💙


He was on his black Apollo mountain bike, fully dressed, wearing a helmet and carrying a backpack containing his laptop and 2 books with his name on them. He also had his mobile phone with him.

On the 27th of June. Noah's naked body was sadly discovered 950m inside a storm drain, between access points. This storm drain was accessible through an area completely unfamiliar to him, behind houses at Northwood Road. https://t.co/bpz3Rmc0wq


"Noah's body was found by specially trained police officers between two drain access points within a section of the tunnel running under the Translink access road," said Mr McCrisken."

Noah's bike was also found near a house, behind a car, in the same area. It had been there for more than 24 hours before a member of public who lived in the street said she read reports of a missing child and checked the bike and phoned the police.
CRIME