NLU: The Power of Spark NLP, the Simplicity of Python

John Snow Labs' NLU is a Python library for applying state-of-the-art text mining, directly on any dataframe, with a single line of code. As a facade of the award-winning Spark NLP library, it comes with 1000+ of pretrained models in 100+, all production-grade, scalable, and trainable, with everything in 1 line of code.

NLU in Action

See how easy it is to use any of the thousands of models in 1 line of code, there are hundreds of tutorials and simple examples you can copy and paste into your projects to achieve State Of The Art easily. <img src="http://ckl-it.de/wp-content/uploads/2020/08/My-Video6.gif" width="1800" height="500"/>

NLU & Streamlit in Action

This 1 line let's you visualize and play with 1000+ SOTA NLU & NLP models in 200 languages

streamlit run https://raw.githubusercontent.com/JohnSnowLabs/nlu/master/examples/streamlit/01_dashboard.py

NLU provides tight and simple integration into Streamlit, which enables building powerful webapps in just 1 line of code which showcase the. View the NLU&Streamlit documentation or NLU & Streamlit examples section. The entire GIF demo and

All NLU resources overview

Take a look at our official NLU page: https://nlu.johnsnowlabs.com/ for user documentation and examples

Ressource	Description
Install NLU	Just run `pip install nlu pyspark==3.0.2`
The NLU Namespace	Find all the names of models you can load with `nlu.load()`
The `nlu.load(<Model>)` function	Load any of the 1000+ models in 1 line
The `nlu.load(<Model>).predict(data)` function	Predict on `Strings`, `List of Strings`, `Numpy Arrays`, `Pandas`, `Modin` and `Spark Dataframes`
The `nlu.load(<train.Model>).fit(data)` function	Train a text classifier for `2-Class`, `N-Classes` `Multi-N-Classes`, `Named-Entitiy-Recognition` or `Parts of Speech Tagging`
The `nlu.load(<Model>).viz(data)` function	Visualize the results of `Word Embedding Similarity Matrix`, `Named Entity Recognizers`, `Dependency Trees & Parts of Speech`, `Entity Resolution`,`Entity Linking` or `Entity Status Assertion`
The `nlu.load(<Model>).viz_streamlit(data)` function	Display an interactive GUI which lets you explore and test every model and feature in NLU in 1 click.
General Concepts	General concepts in NLU
The latest release notes	Newest features added to NLU
Overview NLU 1-liners examples	Most common used models and their results
Overview NLU 1-liners examples for healthcare models	Most common used healthcare models and their results
Overview of all NLU tutorials and Examples	100+ tutorials on how to use NLU on text datasets for various problems and from various sources like Twitter, Chinese News, Crypto News Headlines, Airline Traffic communication, Product review classifier training,
Connect with us on Slack	Problems, questions or suggestions? We have a very active and helpful community of over 2000+ AI enthusiasts putting NLU, Spark NLP & Spark OCR to good use
Discussion Forum	More indepth discussion with the community? Post a thread in our discussion Forum
John Snow Labs Medium	Articles and Tutorials on the NLU, Spark NLP and Spark OCR
John Snow Labs Youtube	Videos and Tutorials on the NLU, Spark NLP and Spark OCR
NLU Website	The official NLU website
Github Issues	Report a bug

Getting Started with NLU

To get your hands on the power of NLU, you just need to install it via pip and ensure Java 8 is installed and properly configured. Checkout Quickstart for more infos

pip install nlu pyspark==3.0.2

Loading and predicting with any model in 1 line python

import nlu 
nlu.load('sentiment').predict('I love NLU! <3')

Loading and predicting with multiple models in 1 line

Get 6 different embeddings in 1 line and use them for downstream data science tasks!

nlu.load('bert elmo albert xlnet glove use').predict('I love NLU! <3')

What kind of models does NLU provide?

NLU provides everything a data scientist might want to wish for in one line of code!

NLU provides everything a data scientist might want to wish for in one line of code!
1000 + pre-trained models
100+ of the latest NLP word embeddings ( BERT, ELMO, ALBERT, XLNET, GLOVE, BIOBERT, ELECTRA, COVIDBERT) and different variations of them
50+ of the latest NLP sentence embeddings ( BERT, ELECTRA, USE) and different variations of them
100+ Classifiers (NER, POS, Emotion, Sarcasm, Questions, Spam)
300+ Supported Languages
Summarize Text and Answer Questions with T5
Labeled and Unlabeled Dependency parsing
Various Text Cleaning and Pre-Processing methods like Stemming, Lemmatizing, Normalizing, Filtering, Cleaning pipelines and more

Classifiers trained on many different datasets

Choose the right tool for the right task! Whether you analyze movies or twitter, NLU has the right model for you!

trec6 classifier
trec10 classifier
spam classifier
fake news classifier
emotion classifier
cyberbullying classifier
sarcasm classifier
sentiment classifier for movies
IMDB Movie Sentiment classifier
Twitter sentiment classifier
NER pretrained on ONTO notes
NER trainer on CONLL
Language classifier for 20 languages on the wiki 20 lang dataset.

Utilities for the Data Science NLU applications

Working with text data can sometimes be quite a dirty job. NLU helps you keep your hands clean by providing components that take away from data engineering intensive tasks.

Datetime Matcher
Pattern Matcher
Chunk Matcher
Phrases Matcher
Stopword Cleaners
Pattern Cleaners
Slang Cleaner

Where can I see all models available in NLU?

For NLU models to load, see the NLU Namespace or the John Snow Labs Modelshub or go straight to the source.

Supported Data Types

Pandas DataFrame and Series
Spark DataFrames
Modin with Ray backend
Modin with Dask backend
Numpy arrays
Strings and lists of strings

Overview of all tutorials using the NLU-Library

In the following tabular, all available tutorials using NLU are listed. These tutorials will help you learn the usage of the NLU library and on how to use it for your own tasks. Some of the tasks NLU does are translating from any language to the english language, lemmatizing, tokenizing, cleaning text from Symbol or unwanted syntax, spellchecking, detecting entities, analyzing sentiments and many more!

{:.table2}

Tutorial Description	NLU Spells Used	Open In Colab	Dataset and Paper References
Albert Word Embeddings with NLU	`albert`, `sentiment pos albert emotion`		Albert-Paper, Albert on Github, Albert on TensorFlow, T-SNE, T-SNE-Albert, Albert_Embedding
Bert Word Embeddings with NLU	`bert`, `pos sentiment emotion bert`		Bert-Paper, Bert Github, T-SNE, T-SNE-Bert, Bert_Embedding
BIOBERT Word Embeddings with NLU	`biobert` , `sentiment pos biobert emotion`		BioBert-Paper, Bert Github , BERT: Deep Bidirectional Transformers, Bert Github, T-SNE, T-SNE-Biobert, Biobert_Embedding
COVIDBERT Word Embeddings with NLU	`covidbert`, `sentiment covidbert pos`		CovidBert-Paper, Bert Github, T-SNE, T-SNE-CovidBert, Covidbert_Embedding
ELECTRA Word Embeddings with NLU	`electra`, `sentiment pos en.embed.electra emotion`		Electra-Paper, T-SNE, T-SNE-Electra, Electra_Embedding
ELMO Word Embeddings with NLU	`elmo`, `sentiment pos elmo emotion`		ELMO-Paper, Elmo-TensorFlow, T-SNE, T-SNE-Elmo, Elmo-Embedding
GLOVE Word Embeddings with NLU	`glove`, `sentiment pos glove emotion`		Glove-Paper, T-SNE, T-SNE-Glove , Glove_Embedding
XLNET Word Embeddings with NLU	`xlnet`, `sentiment pos xlnet emotion`	[![Open In