Challenges of language modeling

Concetti sui Large Language Models (LLM)

Vidhi Chugh

AI strategist and ethicist

Sequence matters!

 

  • I only follow a healthy lifestyle.

 

 

  • Different positions = different meanings

 

  • Only I follow a healthy lifestyle.

Illustration of a person with two arrows in opposite directions and a question mark above their head.

1 Freepik
Concetti sui Large Language Models (LLM)

Context modeling

An image showing the word 'Run' can have multiple meanings depending on the context.

Concetti sui Large Language Models (LLM)

Context modeling

An image showing the word 'Run' to mean jogging.

Concetti sui Large Language Models (LLM)

Context modeling

An image showing the word 'Run' to mean managing or organizing.

Concetti sui Large Language Models (LLM)

Context modeling

An image showing the word 'Run' to mean operating a machine

Concetti sui Large Language Models (LLM)

Context modeling

Adding more context to the previous image by supporting different contextual meanings with the help of examples.

Concetti sui Large Language Models (LLM)

Long-range dependency

 

  • Recognize and connect distant words in a sentence
  • Challenging for traditional language models

An example of long range dependency: "The book that the young girl, who had just returned from her vacation, carefully placed on the shelf was quite heavy."

Concetti sui Large Language Models (LLM)

Single-task learning

An image showing three examples of single-task learning such as image captioning, text summarization, and language translation.

  • Time and resource expensive
  • Less flexible compared to modern LLMs
Concetti sui Large Language Models (LLM)

Multi-task learning

An image showing multi-task learning combining multiple capabilities into one model.

  • Improved performance on each individual task
  • Might impact accuracy and efficiency
  • Less training data needed because data is shared
Concetti sui Large Language Models (LLM)

To recap

Challenges of language modeling:

  • Word sequences

 

  • Understanding context

 

  • Long-range dependency

Single-task learning:

  • Task-specific
  • Less flexible
  • Traditional models and early LLMs

 

Multi-task learning:

  • Versatile
  • Multiple tasks
  • More developed LLMs
Concetti sui Large Language Models (LLM)

Let's practice!

Concetti sui Large Language Models (LLM)

Preparing Video For Download...