What are /r/LanguageTechnology's favorite Products & Services?

From 3.5 billion Reddit comments

I'm a not an OCR expert either, but I've used tesseract a few times and it's quite impressive. Of course the ocr will not be 100% perfect but if your input is a good quality picture, and the handwriting is ~ok, you should have something to work with.

"From scratch" is a big project. What I'd recommend you consider instead is to make a contribution to an existing TTS project. Mycroft's Mimic leaps to mind: Mycroft is an audacious project to make an open source virtual assistant platform and is (mostly) written in Python. https://mycroft.ai/documentation/mimic/ -- scoping your project to contribute something small but meaningful to an existing project can be extremely rewarding and measurable.

There's this article on Neural net language models by Dr. Yoshua Bengio of the University of Montreal, one of the big names in Deep Learning.

Hi,

The phontron link seems to discuss a very limited subset of stuff in NLP.

Secondly, I haven't noticed a need for a lot of math when starting NLP. So you can learn as you go. At some point though, it would be good if you know probabilistic graphical models.

As for a good course: This would introduce you to the field with assignments: https://www.coursera.org/course/nlangp (has some prereq math, study that) You can then read Manning's book and/or go through this other course which covers deep learning's use in NLP: http://cs224d.stanford.edu/syllabus.html

There's also an online course by J&M, https://www.coursera.org/course/nlp, which is based on the textbook and has programming assignments. It's run a few years ago and is now archived, but the videos are still available, and I can share the assignments, if you'd like.

Coursera is offering another NLP course -- this time taught by Michael Collins, but looking at the syllabus I can't imagine it will be any more accessible than the Jurafsky and Martin text.

You're just in time:

> * What is the coolest thing I'll learn if I take this class?

> You will learn how a neural network can generate a plausible completion of almost any sentence.

https://www.coursera.org/course/neuralnets

The method you describe sounds like ad-hoc anomaly detection, so you might want to read up on anomaly detection methods in general.

I have looked into this before but had difficulty following the literature on it. I think this topic might have been called "emerging trend detection" in the past but is now often folded into "event detection". Recent research often focuses on finding events on Twitter, often as they occur rather than retrospectively. "Event extraction" is related, but not primarily what you're looking for (or what I was looking for), and is basically a specialized form of information extraction.

Here's an old survey from 2002 that uses the term "emerging trend detection".

https://www.semanticscholar.org/paper/A-Survey-of-Emerging-Trend-Detection-in-Textual-Kontostathis-Galitsky/76c33146e5d42acd647f1b9648c8b644b8a81710?citationRankingModelVersion=v0.2.0-0.1&year%5B0%5D=2015&year%5B1%5D=2020&sort=total-ci...

Here's a more recent survey from 2018 that also focuses on "trends" and looks promising.

https://www.semanticscholar.org/paper/A-systematic-literature-review-of-mining-weak-and-M%C3%BChlroth-Grottke/45c52a6e76372c1d96ed864c27fade8a8eb0c90a

Hope that helps.

Just to add to the other good recommendations here, Machine Learning is a must. I would say about 75% of NLP papers published today involve ML methods in some way or another. I found this course on Coursera did a very good job of explaining basics and beyond, even though it doesn't talk about NLP specifically.

​

Ok, so, we are talking about www.50gameslike.com

Give it a try, the results are quite good, in my opinion. The biggest the game is, the more precise the results are, because more features has been extracted from text talking about the game.

I build this site because I could not found up to date video recommendations, or because the automatic recommendations sucked: “games like call of duty” would return every call of duty ever made, or “the best RPG on PC” would have 10 years old games in it. 50Gameslike returns current games, out in the last 3 years, and only includes the last / best game of a serie, so that if you are looking for games like call of duty, you will only find one call of duty in the result list,

I would not talk about the recommendation part, that uses what the NER found, it is a different subject,

Right now, there is too much video games every day. You cannot maintain with only 2 hands such a recommendation engine, you have to automate it. Also, when a catalog is crowd maintained, every one will have a slightly different opinion to describe a video game experience. So the solution would be to describe automatically a video game. This is where the NER is used. It takes English text talking about a game, and extract key words ( tags) describing this game: “RPG”, “local couch multiplayer”, etc

If you use a regex to find tags, you have to have a closed list with only the main ones, will have false positive and you cannot have the new trends like “battle royal” or “auto chess”.

​

Ask me if you have particular questions.

The Google Books dataset that drives Google Ngrams Viewer is available for download, but you will probably need to get that uploaded back into a BigQuery like infrastructure if you want to find the n-most frequent instances.

What platform are you targeting? I'll assume PC.

For speech-to-text:

• for Java, there is Sphinx

• for .NET, there's Microsoft Speech SDK (Windows only)

Both pretty easy to use.

Now the question is what kinds of text do you want (free-form dictation vs grammar-based) and what do you want to do with the acquired text.

As you already know that this problem can easily be converted into NER problem. To do that some of the best possible approaches are CRF based token classification or most recent one BERT based token classification. (As mentioned in the other comments as well).

Do have a look at this pytorch article for CRF one, it will help you understand it better. https://pytorch.org/tutorials/beginner/nlp/advanced_tutorial.html To handle your continuous words for a label, we use BIO approach while passing data to the model. (If you don't know, this article should help you).

Suggestion: If you have huge vocabulary, then build a smaller one using sub-word tokenization approaches (BPE, word piece etc) and then train CRF model to learn embedding for those subwords. (if embeddings are not readily available for your domain of data) https://huggingface.co/blog/how-to-train This article would help you regarding how to build subword tokenizer.

Thanks for the question. The main libraries that Trankit's using are pytorch and adapter-transformers. For the GPU requirement, we have tested our toolkit on different scenarios and found that a single GPU with 4GB of memory would be enough for a comfortable use.

Fairly good. It seems the algorithm has improved since you posted. I tried it on this article before, but the extract then seemed too brief and trivial. Now however it does seem to extract several of the important recommendations, though not the final one (last sentence).

I'm not sure I understand what the sentence scrores are based on though. Average proximity of frequently co-ocurring words to one another?

Haven't worked on such a task. But could this be a good starting point for searching popular work in this space? Look at references and citations.

https://www.semanticscholar.org/paper/Hashtag-recommendation-for-short-social-media-texts-Kumar-Baskaran/f1329ca59423b1a038b0dfc3231b5090899106c4#citing-papers

There's been some research on generating useful regexes for various tasks using genetic programming over the years.

https://www.semanticscholar.org/search?q=genetic+regular+expressions&sort=relevance

You can start here: https://www.freecodecamp.org/news/pytorch-full-course/ that has a link to YouTube course that teaches PyTorch.

There are several posts, videos on YouTube and dedicated courses for PyTorch in Coursera and Udemy.

If you are open to doing more studying you could go for an MA or Ph.D. in applied linguistics. My program has two required courses on corpus linguistics and additional ones offered as electives. I am sure that other applied linguistics programs also offer a corpus linguistics specialization.
Otherwise, it might be hard to study it independently because it is fairly complex if you want to do it right. It is a whole science after all. If you want to try, here is a good resource to check out written by the big names in the field: Amazon link.

Very interesting, the more No Code information the better.
Thanks for the insights, we love this kind of article in our team, we also wrote on the subject and we'd love your feedback:

https://zenkit.com/en/blog/what-is-no-code-a-guide/

Arch Linux has a package called words (which you can manually download from here, for example) that has a list of American English words. Names are capitalized. You have both "Amber" and "amber" in the list. If you find a capitalized word that doesn't have a lowercase variant, then it maybe a candidate for a person's name. You could gather such words and compare against your list of names.

Things like handling negation, detecting phrases, misspellings, user intent, vertical intent prediction and more are several of the interesting areas in this. Check out TWIML podcast on NLP in e-commerce engines with Twiggle. I have learnt a lot from several podcasts- which I cover in the list here https://anchor.fm/the-data-life-podcast/episodes/The-Top-5-Data-Science-Podcasts-e3mpem cheers!

There is some research about generating noise artificially. The trick is to transform clean text into images where the text is the same font as the images you usually get, then apply some image noise to it, then OCR it. You then have "genuine" OCR errors, the artificial noise was on the image side and earlier on the pipeline. Since it's 100% artificial you can generate as much as you want.

IIRC this was done mostly for historical texts by researchers of the Université de La Rochelle, eg https://www.semanticscholar.org/paper/An-Analysis-of-the-Performance-of-Named-Entity-over-Hamdi-Jean-Caurant/c39645edca9f27d358f4a6e60f00fc7d86782f5b

Ahmed Hamdi has other papers with this technique too ( https://www.semanticscholar.org/paper/Assessing-and-Minimizing-the-Impact-of-OCR-Quality-Hamdi-Jean-Caurant/bf61cd54d087075f792f8ae7b0a9697af12d5f5f )

PS: I'm not Ahmed Hamdi nor someone from Uni La Rochelle

There actually was a paper on this earlier this year.

https://www.semanticscholar.org/paper/Predicting-Performance-for-Natural-Language-Tasks-Xia-Anastasopoulos/deedb9b61a01d686b28e6034770fccc142e77fab

That said, I think this depends a lot on what kind of task you're doing.

Machine translation is all about large quantities of data. The proper dataset to train a good system for translating a philosophy text from German to English is a parallel corpus (text in both languages aligned on a sentence level) for literally every book and article ever written that's somewhat related to philosophy and exists in both German and English; and it's plausible that this data is too small for good results, so it would be augmtened with general-domain data as well. Anything you can do with a limited amount of training data (e.g. dictionaries but no large parallel corpus) would likely be worse than a state of art general purpose model (e.g. Google Translate), so it may be an interesting learning exercise but no practical benefit - IMHO if you just want to translate a particular book, you should just run it through the best currently existing model you can find and spend the effort on correcting the translation instead of trying to make a better model.

If you do want to play with MT, the existing open source MT engines may be helpful. Some years ago Moses (http://www.statmt.org/moses/) was the go-to solution, but nowadays neural approaches are state of art, so perhaps https://opennmt.net/ or something like that would be usable.

There are some approaches that integrate terminology dictionaries to improve accuracy, but word level translations generally are not the main problem in machine translation currently, it's the larger phrase and sentence structures, disambiguation and fluency.

Thank you for your answer. Blockchain as it is, is a data structure that stores values. So yea, it is used for storing data. Do you mean something like IPFS? Decentralized web? Or browser on top of, for example, ethereum where people can look for information? Sounds interesting :-)

Hi u/ruff, that's great - I'm sure our solution would help with that headache :) Schedule a time here for a 15 minute user test and we'll be glad to provide a beta license for you afterwards. https://calendly.com/humanfirstaicalendar/user-test

Hi Ian,
Some construcctive (I hope) feedback

>EntityKB can be thought of as a reverse search engine. With a typical search engine, such as Solr or Elasticsearch, a corpus of documents is loaded into the system and queries, composed of a single keyword or phrase, when executed against the store return a set of matching documents sorted by relevancy. With EntityKB, a corpus of entities are loaded or programmed into system and queries, composed of the complete text of a document, when executed return a set of matching entities sorted by token position.

This sounds a lot like Elasticsearch percolators percoloatorsand the examples in the docs look a lot like Flashtext / Ahocorasick.

So it's not clear to the end user what this is, or why we should give a new and unproven library a try when there are established solutions out there.

I'd like to see the README start with a motivation example, and show me what this solves that is hard to do somewhere else. The people with that problem will see it and say "wow this is cool" and will be more qualified to give you useful feedback.

The (contents of the) folder that the plugin created should be uploaded somewhere. If you're a student, your university might provide free hosting. Then you just need to have the web address to your html file. Or, if you call the html file 'index.html', the address to the folder. (Browsers know that the index file is the one that they should render, so then it happens automatically.) Otherwise, you need to host the folder somewhere else.

There used to also be free hosting services (as a way to advertise paid services). I don't know where you could get free hosting online anymore. There might still be some services around, but Googling them I don't know which ones are for real and which ones are shady.

EDIT: Make a GitHub page! See https://pages.github.com/ for instructions.

http://wordnet.princeton.edu and related do this, and put homonyms into an ontological hierarchy.

while this is a fantastic project, I have found it to be too incomplete for anything non-trivial. i.e. I tried to recover a common topic from terms like (gun, soldier, bullet, tank) hoping to come out at "war", but clearly that would be too easy ;)

there are some other projects that combine with wordnet or build on top of it, but i'm having a hard time finding them on my phone right now ;)

This task is generally called 'languge modeling', with the most common technical definition being estimating the probability distribution of the next word given the previous context.

The simplest methods are n-gram counts. You might want to read the Jurafsky-Martin book chapter 3 on them - https://web.stanford.edu/~jurafsky/slp3/ ; some tools to do that (build and query language models) are mentioned here http://www.statmt.org/moses/?n=FactoredTraining.BuildingLanguageModel

It's worth noting that 1500 sentences is a tiny toy corpus that by itself is not sufficient to get a meaningful next word prediction. You can get something interesting by taking all the works of an author or all the speeches of a prolific politician, but for 1500 sentences you're not going to get anything nice unless you somehow incorporate a large corpus of general language.

I think your best bet is with tools that do OCR and put the translated text directly above the original image/file (similar to how Google Translate performs live image translation).

Here's a screenshot of what the Yandex Translator OCR was able to do with the picture you posted: Link. The quality of translation isn't as good as Google Translate of course.

If you go the path of parsing a PDF file to extract the text and images and trying to maintain the structure then you're off to some disappointment, specially since your use case's format (magazines) is so flexible and non standardised. Just correctly parsing a multiple-column text on PDF can be quite challenging.

Thanks. Yeah discrete nature will be a problem. Found https://sci-hub.do/downloads/2020-09-04/ba/li2018.pdf that uses nearest neighbors to get closest word for paraphrase generation. But Will go through the references from paper.

You just need a C++ deep learning library and a pre-trained model. I don't know about TF but pytorch has torchscript for C++: https://pytorch.org/tutorials/advanced/cpp_export.html. So you could convert some USE model for pytorch like in the sentence_transformers library to torchscript and go from there.

Look at the init function of the encoder in code block 9. An embedding layer is constructed there, it is mathematically equivalent to the one-hot vector fed to a fully-connected layer and is appropriately updated with backprop. During the forward pass, a sequence of integers (indices to a vocabulary, see block 7) is given to the embedding layer via self.embeddings(src).

Using pre-trained word-embeddings such as word2vec or glove, you would want to initialize it with word2vec vectors instead of using a random initialization, and probably freezing them as well.

The torch documentation is pretty good, take a look at https://pytorch.org/docs/stable/nn.html#embedding .

I've tried increasing the hidden size as well as preloading fasttext vectors instead of randomly initializing embeddings.

But the model keeps performing poorly, using a batchsize of 1 and training on 1 sample, the model quickly diverges to about 0.

But as soon as I increase the batch size a little (e.g. 4), I'm unable to make the model converge.

Also, in the Pytorch seq2seq tutorial, a hidden size (as well as embedding size) of 128 is used, also, they don't use multi-layered rnn's, how come the models performs so well?

​

Thanks

Ah, I see. Maybe elasticsearch ngram tokenizer would work in your case. It offers a hosted solution with the whole elastic ecosystem. You can use to import any corpus and check the frequencies you're looking for.

tesseract seems to use language modeling for that, https://github.com/tesseract-ocr/tesseract/issues/654#issuecomment-274574951

Lots of the product reviews of https://www.amazon.com/NIKE-Womens-Lunarglide-Black-White/dp/B07FXYG5KK/ref=sr_1_1?dchild=1&keywords=lunarglide%2B8&qid=1629112494&sr=8-1&th=1 say that pebbles get trapped in the sole. That's the kind of fact I'd like to be able to extract.

Many similarities between English translations, but also differences which shows many important topics get lost in translation. The book has details: https://www.amazon.com/Analyzed-Machine-Learning-Language-Processing/dp/B087SHQLPX

You should read Taku Kudo's book, 形態素解析の理論と実装 (Theory and Implementation of Morphological Analyzers). He's the creator of MeCab and many parts of the implementation are covered in detail. The MeCab source isn't awful but it doesn't have many comments and it's harder to follow.

https://www.amazon.co.jp/%E5%BD%A2%E6%85%8B%E7%B4%A0%E8%A7%A3%E6%9E%90%E3%81%AE%E7%90%86%E8%AB%96%E3%81%A8%E5%AE%9F%E8%A3%85-%E5%AE%9F%E8%B7%B5%E3%83%BB%E8%87%AA%E7%84%B6%E8%A8%80%E8%AA%9E%E5%87%A6%E7%90%86%E3%82%B7%E3%83%AA%E3%83%BC%E3%82%BA-%E5%...

For the lookup it's traditional to use a Double Array Trie; I'm not aware of any other data structure used in nontrivial applications. Kudo's book's explanation is the clearest I've found, especially for building it, but there's also an English explanation online that's similar.

If you have a good explanation of how a double array trie is implemented... I guess you could knock it out in a dedicated weekend? There's several subtle points in it but for the sake of efficiency there's not many moving parts.

If you wanted to implement a whole tokenizer then the double array tree puts you maybe halfway there if MeCab is your target. Modern MeCab-likes such as Kuromoji and Sudachi tend to have other advanced features, like multiple levels of tokenization, that would be extra work.

Have you seen this: https://play.google.com/store/apps/details?id=com.company.rize

​

I made this app and am willing to help point you in the right direction!

​

Check out the app and see what you think!

Not the best book but if you are looking for a book covering some of the Neural Network methods used in NLP this recent book by Yoav Goldberg seems very promising.

Ah, that makes sense. Yup, using any sort of large corpus like that to create a more general document space should help.

I don't know what the best way to visualize the data is. That's actually one of the big challenges with high dimensional vector spaces like this. Once you've got more than three bases you can't really draw it directly. One thing I have played around with is using D3.js to create a force directed graph where the distance between nodes corresponds to the distance between vectors. It wasn't super helpful though. However I just went to look at some D3.js examples and it looks like there's an example of an adjacency matrix here: https://bost.ocks.org/mike/miserables/ I've never used one, but it seems like it could be helpful.

The link seems to working now for me, but if it stops working again here's the book it was taken from: https://www.amazon.com/Speech-Language-Processing-Daniel-Jurafsky/dp/0131873210 googling the title should help you find some relevant PDFs.

Just happened to have the book open while answering another question, so here is a resource I would suggest: Speech and Language Processing by Jurafsky and Martin, specifically Chapter 23, Section 2 (p. 778): Factoid Question Answering. You could also try Googling "factoid question answering" and "automated factoid question answering" and you'll get a lot of interesting articles. Hope that helps!