Project 2

Introduction

Algorithm of encode

Algorithm of decode

Provided Files

How to get these files

How to submit your programs

Grading Scheme

FAQ

In this project we will manipulate a non-trivial data structure in STL and C++. The data structure in question is a Huffman tree for data compression. A description of a Huffman tree can be found in chapter 22 of Sedgewick's Algorithms textbook.

You will modify some given files, and provide a static Huffman encoder and decoder for files, similar to the pack and unpack utilities. The encoder will take a given file and produce a Huffman-code compressed version of it on the standard output. The decoder will take a given compressed file and produce its uncompressed version on the standard output. There are example executables, encode.sun and decode.sun among the files provided.

The Huffman encoder is static in that it makes two passes over the input file, one to determine the number of occurrences of every character, and a second to generate the Huffman-encoded output. Therefore the output file is of the form:

Field		Description
Header		The string "HuffEnCoded"
CharCount		Number of distinct characters in the file.
Character Table		A CharCount number of entries, each of the form UCHAR Count, where UCHAR is an unsigned char and Count is the number of its occurrence in the Huffman encoded data.
Huffman Data		A stream of binary Huffman-codes of variable length.

1. Make the first pass over the input file. Determine how many different characters are there in the file and their weight ( number of occurrence ). In this step, all the leaf nodes are created, just as the first graph shown in Figure 22.4, P326, Sedgewick Textbook. We need to choose a STL container to contain these nodes.

   In the huffnode.h, we have already given the definition of the class huffnode. Different leaf nodes represent different "chraraters", which is called key of the nodes. Since we don't know how many different charaters will be in the file, the container should be able to increase dynamiclly. After reading a charater from the file, we compare it with the key of the elements now saved in the container. If there is already an element whose key is equal to the character we have just read, we increase the counter of the element, otherwise we create a new element, set this its counter to 1 and add it into the container. Thus the container should be able to use a key to locate a element, returns its possion ( if it exists ). We will use container map ( see Chaper 7.2 and 19.8 of Musser's STL Textbook ) in this project. The declaration will be map<byte,huffnode*,less<byte> >. The member functions we are interested in are "insert()", "find()", etc.

   What should be paid attention is that the container contains pointer to huffnode". we maintain only one copy of each node in the memory, all the other objects or functions use pointer to access them.

2. Built the huffman tree. The idea is easy : choose two subtrees which have the least weight( a single leaf is also a subtree ), merge them into one tree and put it back, repeat this operation until one tree remains. To implement this idea, a new container is needed for two reasons. One is that the key has changeed from "character" to "count". we need to choose element who has the smallest weight ( count ), "character" is not cared. You can vector, list, or other ordinary STL containers, but you have to define a function choose to get the element you want. Or, you can use the generic algorithm to order the elements first, say in increasing order, always choose the first two element , then insert the new one into the proper position to maintain the order of the elements. You can also recommended to use the Priority Queue Container Adaptor ( see Chapter 19.12 of Musser's STL Textbook ). It provides some very useful member functions top(),push(), pop(), which just meet our requirements. The bad news is, to use this Adaptor, a comparison function object needs to be supplied. The good news is, this object has already been provided to you in the file hufftree.h. If you don't want to use it, just leave it there. The other reason for a new container is shown is item 3.
   Don't forget that each leaf has a data member "vector _bits" ( see the definiton of class huffnode in the file huffnode.h ) which representing the code for its "character". This field ( _bits ) can be filled while you buiding the tree. Each time you emerge two subtrees to one, you update this field for all the leaves in this tree. Or you can fill this field after you have built the whole tree.

3. Make the second pass over the input file to encode it. Now the "character" becomes the key again. After reading a character from the file. You can travers the tree to find the leaf which contains this character. But don't forget that we have a great container map. You can use the character you have read as a key to locate the leaf directly without traversing the huffman tree. Since each leaf has a data member _bits, once you locate teh leaf, you get the code you need.

4. Conclusion
   You'd better define a class hufftree which contains the root of the Huffman tree, some useful numbers ( lenth of a file...), the two container we discussed above, several member functions, something like create_leaves_from_file(), build_tree(), encode_file(), output_bits_to_file(),...

1. Read the first part of the encoded file, to make sure it has the string "HuffEnCoded", get the useful numver, and most important, read the Character Table and create the map of leaves.

2. Build the Huffman tree.

3. Read the remaining part of the file, decode the bits.

4. Conclusion
   You have to add some functions to the class hufftree, something like read_leaves_from file(), input_bits_from_file(), decode_file()...

There are five source files provided,

• encode.sun
• decode.sun
• byte.h
• bitstream.h
• huffnode.h
• encode.C
• decode.C
• hufftree.h

You are to modify the last four of these files. There are skeletons provided for all functions and the code is heavily commented, but some functions are left empty on purpose. In those functions there is a comment block starting with //TODO:, and including a description of the required functionality of the function.

byte.h

This short file only contains a definition of type bype.

bitstream.h

This file contains two complete classes, ibitstream and obitstream, which provide bit-per-bit input and output.

huffnode.h

This file provides a near-complete definition of a Huffman-tree node, in the huffnode class. A Huffman-tree node can either be a leaf or an internal node. If it is a leaf it contains the unsigned char it represents, the number of occurrences of that character in the input, and the bits of its corresponding Huffman-code, in the form of an STL vector of bool. If it is an internal node it contains pointer to its left and right child. Each huffnode has a _type which tells whether it is an internal node or a leaf. You are to modify two functions in this file:

• You are to modify the constructor huffnode( huffnode* l, huffnode* r ) so it correctly constructs a new Huffman-tree based on two subtrees.
• You are to provide an operator<< which outputs a Huffman-tree for debugging purposes.

encode.C

This file provides a definition of the Huffman-encoding process, The encoding process opens an input file, reads it over building a mapping to Huffman leaf-nodes as above, then constructs the Huffman tree and finally generates the binary Huffman data using the obitstream class.

decode.C

This file provides a definition of the Huffman-decoding process. The decoding of course only works on input files that are the output of the Huffman-encoder. The decoding process first checks the input file for the header "HuffEnCoded", then reads the the number of characters, the characters themselves and their number of occurrences. It then builds an Huffman-tree just like the one in the encoding process, from the characters and their occurrences, and decodes the Huffman-code binary data in the rest of the file bit-by-bit using the ibitstream class.

hufftree.h

This file provides the definition of the class hufftree. This class contains the most important data structures and functions. It creates all the nodes, both internal and leaf, using a container map to organize all the leaf nodes. It builds the whole Huffman tree, give the Huffman code for every character. It provides some member funtions to read data from or write data to files. You have to spend most of you efforts on this file.

1. Just clik the highlighted <filename>, the contend of the file will appear in your Web browser. Using "Save As" option to save it into your own directory.
2. The place to get these files is the locker assigned to CSC311. After input "add csc_info", you go to the "/ncsu", then "csc_info", then "CSC311", then "LEC", then "003", then "Project2", then you get them.

1. Create a sub-directory with whatever name you like, copy the files into it, and enter this directory to make it current working directory.
2. Enter the command "add csc_info"
3. Enter the command "submit CSC311", then some question will be prompted on your screen. It is not difficult to answer them correctly.

Attention !

      Weight     logical units    
        65%      encode
        30%      decode
         5%      Comments, Styles

      encode will be graded by the following:
      Weight     Description
      	25%      First pass over the file. Using STL class to save data
        25%      Built the Huffman Tree. Using STL class to save datas
        15%      Second pass over the file. Encode it
          
      decode will be graded by the following:
      Weight     Description
        10%      Read the Charter Table correctly
        20%      Decode the file correctly

      comments and styles will be graded by the following
	 3%	 Comments for functions, telling concisely what will be done in them.
	         Comments for important variables, telling what they are used for.
       	 2%      Clear logical structure of your program. 
	         Clear definition of the classes used in your program.