Scala word count program example

 62 total views,  1 views today

Welcome to Scala series. In this tutorial, you will learn how to write a Scala word count program with example.This post also helps you to understand what is MapReduce and how can we use the MapReduce model to write a word count program. Please note, we do not use Apache Spark functionality. This is a plain and simple program using Scala only.

So let us get started..

What is MapReduce?

MapReduce is a programming model  of Hadoop. It is of processing technique used widely by the industry to operate on data parallel.There are two main operations/phases map() and reduce().

In map() phase, we convert the input from one form to another. Say for example, you have list of integers and you want the integers to be converted into String. In this case, we can use map() operation to convert list of integers to list of Strings.

In reduce() phase, we will take input from map() operation, combine the input and produce a single output. When I say combine, we can sum the elements or count number of elements etc.

So we combine call it as MapReduce programming model. Click MapReduce example of Scala to learn more about it.

Scala word count program

In our word count program, we are going to create a list of sentences. We will be running the program to get the number of occurrences of each word.

Below are the operations which we will be using for the word count program. By clicking on the each of the below links you will find more examples of corresponding operation.

Word count program

//Scala word count

package com.scala

object StaticReference {
  def main(args: Array[String]): Unit = {

    //Create a list of sentences
    val list = List(
      "Hey Welcome to StaticReference",
      "Hope you are enjoying my posts in StaticReference site",
      "Hey donot forget to like share and subscribe to our posts")

    //Apply flat map to convert into a list of words
    val words = list.flatMap(line => line.split(" "))

    //group each word
    val groupedData = words.groupBy(item => item)

    //Get the count of each word
    val result = groupedData.mapValues(list => list.size)

    //print
    result.foreach(println)
  }
}

Output

(enjoying,1)
(in,1)
(are,1)
(posts,2)
(our,1)
(site,1)
(donot,1)
(subscribe,1)
(Hey,2)
(to,3)
(you,1)
(share,1)
(my,1)
(forget,1)
(StaticReference,2)
(Welcome,1)
(Hope,1)
(like,1)
(and,1)

This is one of the way to write a word count program. There is alternative way which is given below.

//Scala word count

package com.scala

object StaticReference {
  def main(args: Array[String]): Unit = {

    //Create a list of sentences
    val list = List(
      "Hey Welcome to StaticReference",
      "Hope you are enjoying my posts in StaticReference site",
      "Hey donot forget to like share and subscribe to our posts")

    //Split each sentences into a word by applying flatmap()
    val words = list.flatMap(line => line.split(" "))

    //Map each word
    val keyData = words.map(word => (word, 1))

    //Group each word
    val groupedData = keyData.groupBy(ite => ite._1)

    //Get the sum of values
    val result = groupedData.mapValues(list => {
      list.map(ite => ite._2).sum
    })

    //Print
    result.foreach(println)
  }
}

Output

(enjoying,1)
(in,1)
(are,1)
(posts,2)
(our,1)
(site,1)
(donot,1)
(subscribe,1)
(Hey,2)
(to,3)
(you,1)
(share,1)
(my,1)
(forget,1)
(StaticReference,2)
(Welcome,1)
(Hope,1)
(like,1)
(and,1)

References: 

I hope you like this tutorial and you were able to understand how to write a Scala word count program example

Thanks for reading and please give us a thumb up and comment below!

Tags :

About the Author

Rajasekar

Hey There, My name is Rajasekar and I am the author of this site. I hope you are liking my tutorials and references. Programming and learning new technologies are my passion. The ultimate idea of this site is to share my knowledge(I am still a learner :)) and help you out!. Please spread your words about us (staticreference.com) and give a thumbs up :) Feel free to contact me for any queries!.