[Statlist] Talk on machine learning and NLP - Lausanne - June 27

[Frédéric Ratle]

Dear colleagues,

It is my pleasure to invite you to the following talk on semi-supervised 
learning and
applications in natural language processing:

"Deep Learning: A New Layer" & "NLP: The Brain Way"
Jason Weston
NEC Research

-----
Place:
University of Lausanne (metro UNIL-Sorge),
Amphipôle Building
Room 342 (Geolab)

Time:
14h00 on Friday June 27.

Please do not hesitate to email me for further directions.
-----


Part I
-------------------
Deep Learning: A New Layer
Several techniques for so called "deep learning" of neural networks have 
been recently introduced, e.g. by the groups of Hinton, LeCun and 
Bengio.  We consider these approaches as methods of semi-supervised 
learning and compare them as such to shallow (e.g. SVM-based) 
semi-supervised techniques. We then generalize classical semi-supervised 
techniques (LapSVM and TSVM) to provide neural network "versions" of 
these techniques. We show that these methods work very well and are 
considerably less complicated than existing deep learning techniques.


Part II
--------------------
NLP: The Brain Way
We then apply some of the lessons from the above talk to NLP.
We describe a single convolutional neural network architecture that, 
given a sentence, outputs a host of language processing predictions: 
part-of-speech tags, chunks, named entity tags, semantic roles, 
semantically similar words and the likelihood that the sentence makes 
sense (grammatically and semantically) using a language model. The 
entire network is trained em jointly on all these tasks using 
weight-sharing, an instance of multitask learning.  All the tasks use 
labeled data except the language model which is learnt from unlabeled 
text and represents a novel form of semi-supervised learning for the 
shared tasks. We show how both multitask learning and semi-supervised 
learning improve the generalization of the shared tasks, resulting in 
state-of-the-art performance.



Joint work with Ronan Collobert, Frederic Ratle, Michael Karlen and Ayse 
Erkan.




---

Frédéric Ratle
021 692 35 38