Kotlin Language Features Related to Null Handling

-
Any software engineer with a Java background would find the null handling features in the Kotlin language interesting. Let's summarize this topic with some examples. Nullable types: In Kotlin, types are non-nullable by default. If you want a variable to be able to hold a null value, you need to explicitly declare its type as nullable using the Type? syntax. For example, String? denotes a nullable string, while String represents a non-nullable string. Safe calls (?.): Kotlin introduces the safe call operator (?.) for handling nullable types. It allows you to safely invoke a method or access a property on a nullable object. If the object is null, the expression returns null instead of throwing a NullPointerException. Example: data class Person(val name: String, val age: Int, val address: String?) fun main() {     // Create a person with a nullable address     val person1 = Person("John Doe", 25, "123 Main Street")     val person2 = Person("Jane Doe", 30,
Post a Comment

Image Resizing with Directed Graphs: Seam Carving in Java

-


Seam Carving is an image resizing technique that was discovered in 2007 and it is a feature in Photoshop. We will look at the implementation of this technique using directed graph data structure.

Below are some wikipedia links to the underlying data structre:
https://en.wikipedia.org/wiki/Directed_acyclic_graph
https://en.wikipedia.org/wiki/Shortest_path_problem

The goal is to remove a seam (vertical or horizontal) from the image one by one. To calculate a seam, we need to calculate the energies of the pixels. Let's look at the energy method of our SeamCarver class:

public class SeamCarver {
    private Picture picture;
//...
public double energy(int x, int y) {

         if (x < 0 || x >= width) {
            throw new IllegalArgumentException();
        }
        if (y < 0 || y >= height) {
            throw new IllegalArgumentException();
        }
        if (x == 0 || y == 0 || x == width - 1 || y == height - 1) {
            return 1000.;
        }

         int rx = picture.get(x + 1, y).getRed() - picture.get(x - 1, y).getRed();
        int gx = picture.get(x + 1, y).getGreen() - picture.get(x - 1, y).getGreen();
        int bx = picture.get(x + 1, y).getBlue() - picture.get(x - 1, y).getBlue();

         double deltaSqrx = Math.pow(rx, 2.) + Math.pow(gx, 2.) + Math.pow(bx, 2.);

         int ry = picture.get(x, y + 1).getRed() - picture.get(x, y - 1).getRed();
        int gy = picture.get(x, y + 1).getGreen() - picture.get(x, y - 1).getGreen();
        int by = picture.get(x, y + 1).getBlue() - picture.get(x, y - 1).getBlue();

         double deltaSqry = Math.pow(ry, 2.) + Math.pow(gy, 2.) + Math.pow(by, 2.);
        return Math.sqrt(deltaSqrx + deltaSqry);
    }

So the energy of a pixel is about the RGB values and this is used to find a seam to remove from the image. This is the calculation that allows the content-aware resizing.



We see above the original image and lets see what the energies look like:


The red pixels are the pixels of a vertical seam. That seam will be removed from the image to shrink its width.

Now lets see an example vertical seam and the energies that are calculated:


As we see, we need to find the shortest distance (minimum total energy) from top pixels to the bottom pixels. To do this, we need to model the problem as finding a shortest path in a directed acyclic graph.  So what are the directed edges between the pixels? There will be an edge between a pixel on the top and 3 neighbouring pixels. The weight will be the energy of the target pixel. What we can do is using a directed graph library:

public class SeamCarver {

     private Picture picture;
    private EdgeWeightedDigraph digraphVertical;

     public SeamCarver(
            Picture picture) {                // create a seam carver object based on the given picture
        this.picture = new Picture(picture);
        initVertical(picture);
    }

     private void initVertical(Picture picture) {
        digraphVertical = new EdgeWeightedDigraph(picture.width() * picture.height() + 1);
        for (int col = 0; col < picture.width(); ++col) {
            for (int row = 0; row < picture.height(); ++row) {
                if (row + 1 >= picture.height()) {
                    continue;
                }
                if (col - 1 >= 0) {
                    digraphVertical.addEdge(new DirectedEdge(vertex(col, row), vertex(col - 1, row + 1),       energy(col - 1, row + 1)));
                }
                digraphVertical.addEdge(new DirectedEdge(vertex(col, row), vertex(col, row + 1), energy(col, row + 1)));

                 if (col + 1 < picture.width()) {
                    digraphVertical.addEdge(new DirectedEdge(vertex(col, row), vertex(col + 1, row + 1),     energy(col + 1, row + 1)));
                }
            }
        }

Here we created a graph object with vertices as pixels and created directed edges between pixels. Now we need to use the shortest path algorithm on this graph to find the vertical seams:

    public int[] findVerticalSeam() {  
        double minDist = Double.MAX_VALUE;
        Iterable result = null;
        for (int col = 0; col < picture.width(); ++col) {
            int row = 0; // top
            AcyclicSP acyclicSP = new AcyclicSP(digraphVertical, vertex(col, row));
            for (int botCol = 0; botCol < picture.width(); ++botCol) {
                int botRow = picture.height() - 1;
                double curDist = acyclicSP.distTo(vertex(botCol, botRow));
                if (curDist < minDist) {
                    minDist = curDist;
                    result = acyclicSP.pathTo(vertex(botCol, botRow));
                }
            }
        }
        int[] seam = new int[picture.height()];
        int i = 0;
        for (DirectedEdge edge : result) {
            // System.out.println(edge.from() + "->" + edge.to());
            if (i == 0) {
                seam[i++] = colFromVertex(edge.from());
                seam[i++] = colFromVertex(edge.to());
            } else {
                seam[i++] = colFromVertex(edge.to());
            }
        }
        return seam;
    }

Removing this calculated seam from the image is not difficult:

    public void removeVerticalSeam(int[] seam) {    
        validate(seam);
        Picture picture2 = new Picture(width - 1, height);
        for (int row = 0; row < picture2.height(); ++row) {
            for (int col = 0; col < picture2.width(); ++col) {
                int seamItem = seam[row];
                if (col + 1 >= picture.width()) {
                    continue;
                }
                if (col >= seamItem) {
                    picture2.set(col, row, picture.get(col + 1, row));
                }
                else {
                    picture2.set(col, row, picture.get(col, row));
                }
            }
        }

         this.picture = picture2;
        initialize(this.picture);
    }

Now this solution that uses a graph library works correctly but the performance is not good. Image size can be huge and we need to write our own algorithm implementation on pixels. So we will not use the library classes AcyclicSP and EdgeWeightedDigraph.

public class SeamCarver {

     private Picture picture;
    private double[] vertexEnergy;

     public SeamCarver(Picture picture) {
        if (picture == null) {
            throw new IllegalArgumentException();
        }
        this.picture = new Picture(picture);
        initialize(picture);
    }

     private void initialize(Picture picture) {
        this.width = picture.width();
        this.height = picture.height();

         vertexEnergy = new double[width * height + 1];
        for (int col = 0; col < picture.width(); ++col) {
            for (int row = 0; row < picture.height(); ++row) {
                double energy = energy(col, row);
                vertexEnergy[vertex(col, row)] = energy;
            }
        }
    }

Instead of creating a graph with DirectedEdge objects and energies as weights, I am creating an array of pixel energies.

To implement the shortest path algorithm on the pixels, I need edgeTo[] and distTo[] arrays. Then while traversing from top to bottom, I will perform relaxation. Relaxation is the update of the path: when we see a shorter path, we update the distTo[] and edgeTo[] arrays.

    public int[] findVerticalSeam() {        
        double minDist = Double.MAX_VALUE;
        double[] distTo = new double[V];
        int[] edgeTo = new int[V];

         for (int i = 0; i < edgeTo.length; ++i) {
            edgeTo[i] = -1;
        }
        for (int v = 0; v < V; v++)
            distTo[v] = Double.POSITIVE_INFINITY;
        for (int col = 0; col < this.width; ++col) {
            int row = 0;
            int vertex = vertex(col, row);
            distTo[vertex] = vertexEnergy[vertex];
        }

         for (int row = 0; row < this.height; ++row) {
            for (int col = 0; col < this.width; ++col) {
                if (row == this.height - 1) {
                    // bottom row is already calculated
                    continue;
                }

                 // a: col - 1, row + 1
                // b: col, row + 1
                //c: col + 1, row + 1
                int a, b, c;
                int s = vertex(col, row);

                 if (col - 1 >= 0) {
                    a = vertex(col - 1, row + 1);
                    relax(s, a, distTo, edgeTo);
                }
                b = vertex(col, row + 1);
                relax(s, b, distTo, edgeTo);
                if (col + 1 < this.width) {
                    c = vertex(col + 1, row + 1);
                    relax(s, c, distTo, edgeTo);
                }
            }
        }

         Stack result = null;
        for (int botCol = 0; botCol < this.width; ++botCol) {
            int botRow = this.height - 1;
            double curDist = distTo[vertex(botCol, botRow)];
            if (curDist < minDist) {
                minDist = curDist;
                result = pathTo((vertex(botCol, botRow)), edgeTo);
            }
        }

         int[] seam = new int[this.height];
        int i = 0;
        for (Integer vertex : result) {
            // System.out.println(edge.from() + "->" + edge.to());
            seam[i++] = colFromVertex(vertex);
        }
        return seam;
    }

The relax method is very important and it is a concept of shortest path algorithm for weighted directed acyclic graphs.

private void relax(int s, int v, double[] distTo, int[] edgeTo) {

         if (distTo[v] > distTo[s] + vertexEnergy[v]) {
            distTo[v] = distTo[s] + vertexEnergy[v];
            edgeTo[v] = s;
        }
    }

So if the distance between source vertex "s" and the target vertex "v" is smaller than the existing distance to the target, then we update the edgeTo[] and distTo[] arrays. This means we have found a new shortest path.

NOTES: 

  • One of the applications of graph data structure is image processing.
  • Sometimes using library classes is not enough and we need to understand in detail the underlying algorithm and how the library works.
  • Then we can perform optimizations on the working solution, depending on our special needs.
  • We started by using EdgeWeightedDigraph and AcyclicSP from Princeton University, then we wrote our own algorithm on pixels. 
Thanks to Princeton University Department of Computer Science and Kevin Wayne for the wonderful assignments.

Comments

Post a Comment

Popular posts from this blog

Trie Data Structure and Finding Patterns in a Collection of Words

-
Image
 I faced a very hard problem recently about finding substrings based on a collection of patterns.  Example For  words = ["Apple", "Melon", "Orange", "Watermelon"]  and  parts = ["a", "mel", "lon", "el", "An"] , the output should be  findSubstrings(words, parts) = ["Apple", "Me[lon]", "Or[a]nge", "Water[mel]on"] . While  "Watermelon"  contains three substrings from the  parts  array,  "a" ,  "mel" , and  "lon" ,  "mel"  is the longest substring that appears first in the string. Note: Number of words and number of parts can be huge! A brute force method or careless usage of Java String methods would make it impossible to handle a large number of words and parts (patterns). Solution Approach Whenever we want to search patterns inside a string, we should think about the Trie data structure:  https://en.wikipedia.
Read more

swapLexOrder: Finding lexicographically largest string

-
The following coding question is really amazing: Given a string  str  and array of  pairs  that indicates which indices in the string can be swapped, return the  lexicographically largest  string that results from doing the allowed swaps. You can swap indices any number of times. Example For  str = "abdc"  and  pairs = [[1, 4], [3, 4]] , the output should be swapLexOrder(str, pairs) = "dbca" . By swapping the given indices, you get the strings:  "cbda" ,  "cbad" ,  "dbac" ,  "dbca" . The lexicographically largest string in this list is  "dbca" . Input/Output [execution time limit] 3 seconds (java) [input] string str A string consisting only of lowercase English letters. Guaranteed constraints: 1 ≤ str.length ≤ 10 4 . [input] array.array.integer pairs An array containing pairs of indices that can be swapped in  str  (1-based). This means that for each  pairs[i] , you can swap elements in  str  that have the indices  p
Read more

A Graph Application in Java: Using WordNet to Find Outcast Words

-
Image
I'll explain our intent with an example: Let's say we have the following words: horse, zebra, cat, bear, table Which one is the outcast? The table.. But how can a computer system find this? We are talking about the whole English language and any word can be in this list. The number of words in the list can be huge, so the solution should use appropriate data structures and algorithms.  We'll use WordNet which is a graph of English words. It contains the semantic relationships between words and our concern is the "is a" relationship. For our example, we know that horse and cat are both animals. "Table" is not an animal and it is the outcast in this situation. But we even have the ability to tell that horse and zebra is very close. Below is a unit test that shows a high level view of our application in Java: public class OutcastTest {     private WordNet wordNet;     private Outcast outcast;     @Before  
Read more