Data cleaning and transformation

Case Study: Financial Analysis in KNIME

Andrew Logan

Data Scientist, 4&8 Ltd

Sticking to the plan

  • Clean, useful, and consistent data
  • Cleaning: missing values, deduplication, removing unnecessary rows and columns, etc
  • Transformation: string manipulation, numerical calculations, renaming columns etc

KNIME workflow process with Clean and Transform highlighted

Case Study: Financial Analysis in KNIME

Sparkling clean

  • Most common nodes: Missing Value, Duplicate Row Filter, Column Filter, String Cleaner
  • Annotate!

KNIME cleaning nodes

Case Study: Financial Analysis in KNIME

Sparkling clean

  • Most common nodes: Missing Value, Duplicate Row Filter, Column Filter, String Cleaner
  • Annotate!
  • Many others: use the KNIME forum - https://forum.knime.com/ or search engine for special cases

KNIME forum search results for cleaning

Case Study: Financial Analysis in KNIME

Making useful changes

  • Manipulating data to make it useful
  • String manipulation

String Manipulation nodes

Case Study: Financial Analysis in KNIME

Making useful changes

  • Manipulating data to make it useful
  • String manipulation
  • Numerical calculation

Numerical Calculation nodes

Case Study: Financial Analysis in KNIME

Making useful changes

  • Manipulating data to make it useful
  • String manipulation
  • Numerical calculation
  • Some other transformation nodes
  • Annotations!

Some other Transformation nodes

Case Study: Financial Analysis in KNIME

Easier to understand and re-use

  • Metanodes - the towns on the map.

Roadmap illustration

Case Study: Financial Analysis in KNIME

Easier to understand and re-use

  • Metanodes - the towns on the map.
  • Many nodes into one

Creating metanode example gif - multiple nodes into one

Case Study: Financial Analysis in KNIME

Easier to understand and re-use

  • Metanodes - the towns on the map.
  • Many nodes into one
  • Time saver
  • Easy to understand
  • Easy to annotate

Inside a metanode

Case Study: Financial Analysis in KNIME

Let's practice!

Case Study: Financial Analysis in KNIME

Preparing Video For Download...