Insights from an open source influencer

I'm often asked how I get my content, over the years I've built an unusual technology stack for it

https://t.co/FdIAeZc8GF

Some insights:

I use Feedly for most content inputs because I can access the content through a single API endpoint and scraping is often pure pain.

@feedly

Feedly saves me a lot of time and manual

To pull and enrich my GitHub content I use ghapi from @fastdotai which provides a 100% always-updated coverage of the entire GitHub

The actual magic happens on my server infrastructure.

This is where the various data streams converge and are enriched.

The whole thing is based on Kafka streams so that if a pipeline stops it can be started up again without any problems.

Then a wild mix of machine learning NLP pipelines between @huggingface and @spacy_io is used to classify, score and tag the content.

Why are graphs the future of biomedical research and what is the value of NLP here?

A small case study about:

How to speed up drug discovery with knowledge graphs and discover potential cures for diseases

In this case text mining is used to contextualize knowledge about:

- Genes
- Compounds
- Diseases
- Adverse drug effects
- Receptor bindings

Which text types are processed here? Medical literature, patient notes, electronic health records, clinical reports etc.

But how to start?

First you need to identify the different entities such as compounds, diseases, adverse drug effects and receptor bindings.

This is achieved through Natural Language Processing (NLP) and there are suitable pre-trained models for processing biomedical, scientific or clinical text like scispaCy

@spacy_io models for processing biomedical, scientific or clinical

Another library which is specialized in biomedical text is Spark NLP

@JohnSnowLabs