Udai OMR tool

This is a free-to-use OMR (Optimal Mark Recognition) tool, meant for non-proft social development organizations to conduct surveys and digitize data recorded on paper forms. It is especially suited for extracting data from forms that have been photo-copied and then scanned; hence, the resultant images are likely to suffer from rotations, smudge marks, and random lines here and there. Examples of a few forms that the tool is able to handle with 100% accuracy are shown here.
The motivation for writing this tool came from the fact that there is no free OMR software available today, even though it is of great use to social development organizations!

The tool has been written by Aaditeshwar Seth from the Udai Waterloo chapter, at the University of Waterloo. Udai is a volunteer student organization at the University of Waterloo in Canada and the University of California at San Diego.

In case of any questions or clarifications or bugs, please write to Aaditeshwar -- a3seth [AT] uwaterloo [DOT] ca -- with the subject "OMR tool".

Usage

  1. You first make a form in powerpoint using our template, for example, thane.ppt. Make sure that you do not move the two concentric circles in the top-left and bottom-right corners of the template. These are used by the software to determine the angle of rotation of the form.

  2. Take a printout and make as many photocopies of the form as you want.

  3. On the original printout, color all the marks with a black pen, and scan this marked up printout. Please be as careful as you can to align the printout so that there is no rotation. Some amount of error is fine, but not too much! We will call this scanned image the template, shown below.
  4. Run the following:

    java -cp "dist/OMRProj.jar;dist/lib/jiu.jar" omrproj.ProcessTemplate [template-filename]

    This is a one-time operation and can take a while to complete. If all goes well, it will generate 4 files.
    1. [template-filename].asc in the same directory as the template file.
    2. [template-filename].config in the same directory.
    3. marked.png in the current directory.
    4. scaled.png in the current directory.

    Do not touch the .config file! You can delete the .png files if you want.

  5. Open the .asc file in a standard text editor such as write or textpad. This is actually an ascii representation of the template with all the marks marked with a '0', as shown below. 
    ---------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------
--0----0----0----0----0--------0----0----0----0----0---------0----------------0--------
---------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------
-----------------------0--0--0---0--0---0--0---0--0---0--0-----0---0--0---0--0---0-----
---------------------------------------------------------------------------------------
---0--------0----------0--0--0---0--0---0--0---0--0---0--0-----0---0--0---0--0--0------
---------------------------------------------------------------------------------------
----------------------0---0--0---0--0---0--0---0--0--0---0-----0---0--0---0--0--0------
---------------------------------------------------------------------------------------
----0----------0-------0--0--0---0--0---0--0---0--0--0---0-----0---0--0--0---0--0------
---------------------------------------------------------------------------------------
-----------------------0--0---0--0--0---0--0---0--0---0--0-----0---0--0---0--0---0-----
---------------------------------------------------------------------------------------
-----------------------0--0---0--0---0--0--0---0--0---0--0-----0---0--0---0--0---0-----
---------------------------------------------------------------------------------------
-----------------------0--0---0--0--0---0--0---0--0---0--0-----0---0--0---0--0---0-----
---------------------------------------------------------------------------------------
-----------------------0--0---0--0---0--0--0---0--0---0--0-----0---0--0---0--0---0-----
---------------------------------------------------------------------------------------
----0--------0---------0--0---0--0--0---0--0---0--0---0--0-----0---0--0---0--0---0-----
---------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------
-----------------0----------------------0--------------------------0--------------0----
---------------------------------------------------------------------------------------
-----------------0----------------------0--------------------------0--------------0----
---------------------------------------------------------------------------------------
-----------------0----------------------0-----------------------------------------0----
---------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------
-----------------0----------------------0-----------------------------------------0----
---------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------
-----------------0-------------0---------------0-----------------0----------------0----
---------------------------------------------------------------------------------------
-----------------0-------------0---------------0----------------0-----------------0----
---------------------------------------------------------------------------------------
----------------0-----------------------------0------------------0----------------0----
---------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------
------------------0------------0-------------------------------------------------------
---------------------------------------------------------------------------------------
-----------------0-------------0-------------------------------------------------------
---------------------------------------------------------------------------------------
------------------0------------0-------------------------------------------------------
---------------------------------------------------------------------------------------
------------------0--------------------------------------------------------------------
---------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------
    Some of the marks may not be exactly aligned, but don't worry. Now, very carefully, label the marks with single characters, by setting all the marks in the same group with the same character. This is shown below. In case you run out of alphabets, you can use numerals or any other character such as - + ? . etc. Make sure that you do not alter the position of any mark! This is agreably hard to do, but we chose this method to avoid the need for making a graphical frontend. 
    ---------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------
--a----a----a----a----a--------b----b----b----b----b---------c----------------c--------
---------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------
-----------------------f--f--f---f--f---f--f---f--f---f--f-----h---h--h---h--h---h-----
---------------------------------------------------------------------------------------
---d--------d----------f--f--f---f--f---f--f---f--f---f--f-----h---h--h---h--h--h------
---------------------------------------------------------------------------------------
----------------------f---f--f---f--f---f--f---f--f--f---f-----h---h--h---h--h--h------
---------------------------------------------------------------------------------------
----e----------e-------f--f--f---f--f---f--f---f--f--f---f-----h---h--h--h---h--h------
---------------------------------------------------------------------------------------
-----------------------f--f---f--f--f---f--f---f--f---f--f-----h---h--h---h--h---h-----
---------------------------------------------------------------------------------------
-----------------------f--f---f--f---f--f--f---f--f---f--f-----h---h--h---h--h---h-----
---------------------------------------------------------------------------------------
-----------------------f--f---f--f--f---f--f---f--f---f--f-----h---h--h---h--h---h-----
---------------------------------------------------------------------------------------
-----------------------f--f---f--f---f--f--f---f--f---f--f-----h---h--h---h--h---h-----
---------------------------------------------------------------------------------------
----g--------g---------f--f---f--f--f---f--f---f--f---f--f-----h---h--h---h--h---h-----
---------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------
-----------------i----------------------j--------------------------k--------------l----
---------------------------------------------------------------------------------------
-----------------i----------------------j--------------------------k--------------l----
---------------------------------------------------------------------------------------
-----------------i----------------------j-----------------------------------------l----
---------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------
-----------------i----------------------j-----------------------------------------l----
---------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------
-----------------q-------------p---------------o-----------------n----------------m----
---------------------------------------------------------------------------------------
-----------------q-------------p---------------o----------------n-----------------m----
---------------------------------------------------------------------------------------
----------------q-----------------------------o------------------n----------------m----
---------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------
------------------r------------s-------------------------------------------------------
---------------------------------------------------------------------------------------
-----------------r-------------s-------------------------------------------------------
---------------------------------------------------------------------------------------
------------------r------------s-------------------------------------------------------
---------------------------------------------------------------------------------------
------------------r--------------------------------------------------------------------
---------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------

  6. The user-unfriendliness does not end here! You now need to make a [template-filename].fields file, which contains the details of the various form-fields, as follows. 
    a row single yearsInThane 1 2 3 4 5
b row single yearsInSlum 1 2 3 4 5
c row single nameOnElectoral yes no
d row single rationCard yes no
e row single rationCardColor orange yellow
f grid column rationCardNumber 0 1 2 3 4 5 6 7 8
g row single voterId yes no
h grid column dateOfBirth 0 1 2 3 4 5 6 7 8
i column single motherTongue marathi hindi kannada other
j column single houseType kaccha semipukka pukka house
k column single houseOwner own rent
l column single vehicle none 2wh bike other
m column single garbage bin open other
n column single disposal soakpit septic sewage
o column single toilet own public other
p column single water own public
q column single electricity ownmeter borrowed other
r column multiple comforts cable fan fridge newspaper
s column multiple moreComforts telephone mixer tv
    The format of each line is: character type subtype name value1 value2....
    1. Character: This is the same as the character you used in the .asc file
    2. Type and Subtype: The types of fields supported are single and multiple choice lists, aligned row-wise (horizontally) or column-wise (vertically). And grids aligned horizontally or vertically. Each field is denoted by (i) row single or row multiple for a single or multiple selection field aligned row-wise, (ii) column single or column multiple for a single or mutiple selection field aligned column-wise, or (iii) grid row or grid column for a grid field aligned row-wise or column-wise respectively.
    3. Field name: This can be any name you want to associate with the field.
    4. Values: These are the field values, in order.

  7. Now, you are finally all set! Fill out the photocopied forms, scan them, and run the following:

    java -cp "dist/OMRProj.jar;dist/lib/jiu.jar" omrproj.ProcessForm [form-filename] [template-filename]

    If all goes well, this will generate two files.
    1. [form-filename].dat in the same directory as the form file.
    2. marksfoundform.png in the current directory.

    You can delete the png file if you want. It is the same as the form image, but with a small 'x' on the marks that were recognized. And the .dat file actually contains the form data, similar to the following. 
    moreComforts=mixer
comforts=cable
electricity=ownmeter
water=own
toilet=own
disposal=septic
garbage=bin
vehicle=none
houseOwner=own
houseType=semipukka
motherTongue=marathi
dateOfBirth=101475
voterId=no
rationCardNumber=1235545678
rationCardColor=orange
rationCard=no
nameOnElectoral=yes
yearsInSlum=3
yearsInThane=4

Download

Please note that the code has not been tested on images with different resolutions. There are a few hardcoded parameters in a few places which could cause problems. I will try to fix this soon, but until then, if it does not work for you then maybe you will have to tinker with the code a bit!
Source and distribution

Sample files

test.tar.gz contains the following:

Powerpoint: thane.ppt
Template: 2circle-org-colored-whole.tif
ASCII template: 2circle-org-colored-whole.tif.asc
Fields: 2circle-org-colored-whole.tif.fields
Forms: 2circle-1.tif, 2circle-2.tif, 2circle-3.tif, 2circle-4.tif, 2circle-5.tif, 2circle-6.tif, 2circle-7.tif

Limitations

The scanned template and form images should be of the same resolution. This is not a serious limitation though, because the template can be scanned at different resolutionss for different form resolutions.

The code also has to be re-factored and documented. This will get done in good time.

The form data has to be eventually pushed into a database.

Future work

We want to simply use a digital camera instead of a scanner. This will hopefully get done in the next few weeks.

Credits

This software uses the JIU (Java Imaging Utilities) library.

The project was initiated with Aniket Kate and Maheedhar Kolla at the University of Waterloo, to help Shelter-Associates, an NGO based in Pune.