Challenges of language modeling

Large Language Models (LLMs) Concepts

Vidhi Chugh

AI strategist and ethicist

Sequence matters!

 

  • I only follow a healthy lifestyle.

 

 

  • Different positions = different meanings

 

  • Only I follow a healthy lifestyle.

Illustration of a person with two arrows in opposite directions and a question mark above their head.

1 Freepik
Large Language Models (LLMs) Concepts

Context modeling

An image showing the word 'Run' can have multiple meanings depending on the context.

Large Language Models (LLMs) Concepts

Context modeling

An image showing the word 'Run' to mean jogging.

Large Language Models (LLMs) Concepts

Context modeling

An image showing the word 'Run' to mean managing or organizing.

Large Language Models (LLMs) Concepts

Context modeling

An image showing the word 'Run' to mean operating a machine

Large Language Models (LLMs) Concepts

Context modeling

Adding more context to the previous image by supporting different contextual meanings with the help of examples.

Large Language Models (LLMs) Concepts

Long-range dependency

 

  • Recognize and connect distant words in a sentence
  • Challenging for traditional language models

An example of long range dependency: "The book that the young girl, who had just returned from her vacation, carefully placed on the shelf was quite heavy."

Large Language Models (LLMs) Concepts

Single-task learning

An image showing three examples of single-task learning such as image captioning, text summarization, and language translation.

  • Time and resource expensive
  • Less flexible compared to modern LLMs
Large Language Models (LLMs) Concepts

Multi-task learning

An image showing multi-task learning combining multiple capabilities into one model.

  • Improved performance on each individual task
  • Might impact accuracy and efficiency
  • Less training data needed because data is shared
Large Language Models (LLMs) Concepts

To recap

Challenges of language modeling:

  • Word sequences

 

  • Understanding context

 

  • Long-range dependency

Single-task learning:

  • Task-specific
  • Less flexible
  • Traditional models and early LLMs

 

Multi-task learning:

  • Versatile
  • Multiple tasks
  • More developed LLMs
Large Language Models (LLMs) Concepts

Let's practice!

Large Language Models (LLMs) Concepts

Preparing Video For Download...