Tuesday, March 18, 2014

Packing and Obfuscating Javascript Files, and also Deobfuscation

I personally wrote a thesis about Java obfuscation. Really. Anyway, this post would talk about Javascript file packing. There are several benefits of doing the packing process :
  • Minimizing the javascript file size, therefore speeding up the website download time for users using mobile device and less-than-3G bandwidths
  • Obfuscating the content of the javascript file, keeping unwanted eyes or parties (such as your client) from reading your source code. You might have spend much time and effort writing them so you don't want anyone to easily copy and modify your works.
But in some organization, like where I worked, needed to change the source code now and then. Normally we include a contract clause such as these "full source code of the application built must be given to the company that paid for the programming work". But sometimes we forgot to state the clause, or we might forgot to remind the programmer about the clause, leaving us without the original source code, and only two or three days to modify the application before it must went live/production.
This post would discuss about pack/unpack process using Dean Edward's Packer (http://dean.edwards.name/packer/). The obfuscation level is actually pretty weak, and the packing process is more similar to source code encoding technique rather than obfuscation techniques.


Packing Process

To pack a Javascript file, we could use .NET application provided in Dean Edward's site or to use the online packer found in this url. The resulting string could be pasted into a

Maven Hell Again and Again

Something is wrong with Maven. Really. Even though no-1 problem of smooth maven operation in my country is no longer an issue (the no 1 problem is low bandwidth internet connectivity, this no longer an issue since my office's bandwidth has been upgraded tremendously), other issues still plague my maven operation today.


See, I have this one production java application, built using Spring Roo (a good technology choice at the moment) and each time the source code changes I am forced to waste time repairing my maven configuration. The problem is a failed dependency resolution that took a long time to investigate.


One of the previous big time-waster when I am using maven is the problematic Maven-Eclipse integration. I avoid Eclipse now, currently I am using IntelliJ's IDEA Community edition.
The error is something like :

At first I was confused that the problem is about stale local repository cache, which I tried to fix using mvn -U switch. Because it does no good, I checked the milestone repository, I found no jta artifact there. Then I thought, why on earth doesn't maven search the central repository. After all, it supposed to have the artifact (HTML fragment from http://mavenhub.com/mvn/central/javax.transaction/jta/1.0.1B#depMaven ) :

After checking fruitlessly at my user settings.xml and global settings.xml seeking a clue where does the central repository got disabled,  I changed maven version from 3.0.5 to 2.2.2.
It turned out that it already did search the maven's central repository in addition to milestone repository and snapshot repository.  Look at the resulting error message :

From there, I double checked the directory list of http://repo1.maven.org/maven2/javax/transaction/jta/1.0.1B/ :

Index of /maven2/javax/transaction/jta/1.0.1B/


../
jta-1.0.1B.pom                                     08-Nov-2005 21:58                 515
jta-1.0.1B.pom.md5                                 22-Nov-2010 06:05                  33
jta-1.0.1B.pom.sha1                                22-Nov-2010 06:05                  41
maven-metadata.xml                                 06-Nov-2006 01:08                 175
maven-metadata.xml.md5                             22-Nov-2010 06:05                  33
maven-metadata.xml.sha1                            22-Nov-2010 06:05                  41


There is no jar file there, but the pom files are there. I got deceived by a few pom files :). So there lies the actual problem.

It is strange that maven 3.0.5 output a different error message than 2.2.1 :
Maven 3.0.5 build failure message




Compare this with 2.2.1 build failure message :


Maven 2.2.1 error message
The second error message tells me that maven already searched central repository. 
This might be counter intuitive, but yes, it might be useful to have multiple Maven versions in your system. 
So what is the solution ? Just add the java.net repository that still keeps the jta-1.0.1B jar file :
 The solution is courtesy of http://stackoverflow.com/questions/9003298/missing-javax-transaction-jta-artifact. So it might be better to start combing stack overflow for solutions anyway. But you need to use the correct search keywords first, such as [unable find jta repository central]. Such keyword doesn't came to mind easily though.

And I also want to point out that it is not necessary to use the mvn -e switch or the mvn -X switch. The complex output doesn't have additional benefit in this case.


Saturday, March 15, 2014

Practicing for the Google Code Jam - Milkshake

I'm trying to practice for the Google Code Jam. In particular, I want to blog about the Milkshake problem (Round 1A 2008). The problem definition could be read here. My idea is to do a dijkstra graph search algorithm to expand sorted nodes. The node consists of :
  • a state array of the milkshake preparation, * stands for undecided, 0 stands for unmalted, and 1 stands for malted.
  • satisfied customer count
  • satisfied customer boolean array
A customer is said to be satisfied if one of the decided preparation contains one of his/her milkshake flavor preference. The nodes were sorted first by malted flavor count, and then by satisfied customer count. In Dijkstra's terms, we are doing graph traversal on the nodes minimizing cost, whereas the cost is defined as the malted flavor count, and the destination is defined as the condition where all customers are satisfied. The new nodes are defined as a new state where one of the customer's flavors are satisfied. We skip satisfied customers because they have no effect on the world state and objective satisfaction. My first solution failed to solve the small problem set, my basic ideas were not wrong, but inefficient. The source for the first solution is :
package yudhi.pkg;

import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
import java.util.ArrayList;
import java.util.List;
import java.util.PriorityQueue;

/**
 * Created with IntelliJ IDEA.
 * User: yudhi
 * Date: 3/15/14
 * Time: 4:00 PM
 * To change this template use File | Settings | File Templates.
 */
public class Milkshake {
    public Milkshake(int flavorCnt, Customer[] customerList) {

        this.flavorCnt = flavorCnt;
        this.customers = customerList;
    }

    public static void main(String[] args) throws IOException {
        BufferedReader br = new BufferedReader(new InputStreamReader(System.in));
        String testCasesStr= br.readLine();
        int testCases = Integer.parseInt(testCasesStr);
        for (int caseNo=1; caseNo<=testCases; caseNo++)
        {
            String flavorCntStr = br.readLine();
            int flavorCnt = Integer.parseInt(flavorCntStr);
            int custCount = Integer.parseInt(br.readLine());
            Customer[] customerList = new Customer[custCount];
            for (int custIdx=0; custIdx<custCount; custIdx++)
            {
               String custspec = br.readLine();
               String[] words = custspec.split(" ");
               int custFlavorCnt = Integer.parseInt(words[0]);
               int[] likedFlavors = new int[custFlavorCnt];
               boolean[] likeMalted = new boolean[custFlavorCnt];
               for (int i=0; i<custFlavorCnt; i++)
               {
                   likedFlavors[i]= Integer.parseInt(words[i*2+1]);
                   likeMalted[i] = (Integer.parseInt(words[i*2+2]))>0;

               }
               customerList[custIdx] = new Customer(custIdx,custFlavorCnt,likedFlavors,likeMalted);
            }
            Milkshake a = new Milkshake(flavorCnt+1,customerList); // hack because our flavors start from 0
            Node result = a.doDijkstra();
            if (result == null)
            {
                System.out.println("Case #" + caseNo+": IMPOSSIBLE");
            }
            else
            {
                System.out.print("Case #" + caseNo+":");
                for (int j=1; j<= flavorCnt; j++) // start from 1
                {
                    System.out.print(' ');
                    if (result.state[j]=='*') System.out.print('0'); else
                    System.out.print(result.state[j]);
                }
                System.out.println();

            }

        }

    }
    final Customer[] customers;
    final int flavorCnt;
    public Node doDijkstra()
    {
        PriorityQueue<Node> nodesToOpen = new PriorityQueue<Node>();
        nodesToOpen.add(initialNode());
        Node answer = null;
        while (!nodesToOpen.isEmpty())
        {
            Node n = nodesToOpen.poll();
            if (n.satisfiedCnt == customers.length) {
                answer = n;
                break;
            }
            for (int custIdx=0; custIdx< customers.length; custIdx++)
            {
                if (n.satisfiedStates[custIdx]) continue;
                List<Node> newNodes = n.generateNewNodes(customers[custIdx]);
                for (Node newN: newNodes)
                {
                    nodesToOpen.add(newN);
                }
            }
        }
        return answer;
    }
    public static class Customer{
        final   int index;
        final int flavorCnt;
        final int[] likedFlavors;
        final boolean[] likedMalted;

        public Customer(int custIdx, int custFlavorCnt, int[] likedFlavors, boolean[] likeMalted) {

            this.index = custIdx;
            this.flavorCnt= custFlavorCnt;
            this.likedFlavors = likedFlavors;
            this.likedMalted = likeMalted;
        }
    }

    public Node initialNode()
    {
        StringBuilder sb = new StringBuilder();
        for (int i=0; i<flavorCnt; i++ )
            sb.append('*');
        return new Node(sb.toString().toCharArray());
    }

    static char[] changeState(char[] origState, int flavorNo, boolean likeMalted)
    {
        char[] istate = origState.clone();
        if (likeMalted) {
            istate[flavorNo] = '1';
        }  else istate[flavorNo] = '0';
        return istate;
    }
    public class Node implements Comparable<Node> {
        final char[] state;
        final int satisfiedCnt;
        final int maltCost;
        final boolean[] satisfiedStates;
        List<Node> generateNewNodes(Customer c)
        {
            List<Node> r = new ArrayList<Node>();
            if (satisfiedStates[c.index]) return r;
            for (int flavorIdx=0; flavorIdx<c.flavorCnt; flavorIdx++)
            {
                int flavorNo = c.likedFlavors[flavorIdx];
                boolean likeMalted = c.likedMalted[flavorIdx];
                char currentFlavorState = state[flavorNo];
                if (currentFlavorState == '*')
                {
                   char[] newstate = changeState(this.state,flavorNo,likeMalted);
                   r.add(new Node(newstate));
                }
            }
            return r;
        }

        public Node(char[] nstate)
        {
            state = nstate;
            int cntSatisfied=0;
            int cost =0;
            boolean[] satisfiedStates = new boolean[customers.length];
            for (int j=0; j<flavorCnt; j++)
            {
               if (nstate[j]=='1') cost++;
            }
            maltCost = cost;
            for (Customer c : customers)
            {
               boolean thisSatisfied = false;
               for (int i=0; i<c.flavorCnt; i++)
               {
                   int flavorNo = c.likedFlavors[i];
                   boolean likeMalted = c.likedMalted[i];
                   if (likeMalted)
                   {
                        if (state[flavorNo]== '1') thisSatisfied = true;
                   } else {
                       if (state[flavorNo]== '0') thisSatisfied = true;
                   }
               }
                if (thisSatisfied) {
                    cntSatisfied ++;
                    satisfiedStates[c.index] =true;
                } else satisfiedStates[c.index] = false;
            }
            satisfiedCnt = cntSatisfied;
            this.satisfiedStates =satisfiedStates;
        }

        @Override
        public int compareTo(Node o) {
            if (this.maltCost < o.maltCost)
                return -1;
            if (this.maltCost > o.maltCost)
                return 1;
            if (this.satisfiedCnt > o.satisfiedCnt)
                return -1;
            if (this.satisfiedCnt < o.satisfiedCnt)
                return 1;
            return 0;

        }
    }
}




Because of the failure solving first problem set, I changed the algorithm a bit to the second solution, starting from state '0' instead of '*' and allowing changes on already set states :
 
package yudhi.pkg;

import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
import java.util.*;

/**
 * Created with IntelliJ IDEA.
 * User: yudhi
 * Date: 3/15/14
 * Time: 4:00 PM
 * To change this template use File | Settings | File Templates.
 */
public class MilkshakeB {
    public MilkshakeB(int flavorCnt, Customer[] customerList) {

        this.flavorCnt = flavorCnt;
        this.customers = customerList;
    }

    public static void main(String[] args) throws IOException {
        BufferedReader br = new BufferedReader(new InputStreamReader(System.in));
        String testCasesStr= br.readLine();
        int testCases = Integer.parseInt(testCasesStr);
        for (int caseNo=1; caseNo<=testCases; caseNo++)
        {
            String flavorCntStr = br.readLine();
            int flavorCnt = Integer.parseInt(flavorCntStr);
            int custCount = Integer.parseInt(br.readLine());
            Customer[] customerList = new Customer[custCount];
            for (int custIdx=0; custIdx<custCount; custIdx++)
            {
                String custspec = br.readLine();
                String[] words = custspec.split(" ");
                int custFlavorCnt = Integer.parseInt(words[0]);
                int[] likedFlavors = new int[custFlavorCnt];
                boolean[] likeMalted = new boolean[custFlavorCnt];
                for (int i=0; i<custFlavorCnt; i++)
                {
                    likedFlavors[i]= Integer.parseInt(words[i*2+1]);
                    likeMalted[i] = (Integer.parseInt(words[i*2+2]))>0;

                }
                customerList[custIdx] = new Customer(custIdx,custFlavorCnt,likedFlavors,likeMalted);
            }
            MilkshakeB a = new MilkshakeB(flavorCnt+1,customerList); // hack because our flavors start from 0
            Node result = a.doDijkstra();
            if (result == null)
            {
                System.out.println("Case #" + caseNo+": IMPOSSIBLE");
            }
            else
            {
                System.out.print("Case #" + caseNo+":");
                for (int j=1; j<= flavorCnt; j++) // start from 1
                {
                    System.out.print(' ');
                    if (result.state[j]=='*') System.out.print('0'); else
                        System.out.print(result.state[j]);
                }
                System.out.println();

            }

        }

    }
    final Customer[] customers;
    final int flavorCnt;
    public Node doDijkstra()
    {
        PriorityQueue<Node> nodesToOpen = new PriorityQueue<Node>();
        HashSet<Node> knownNodes = new HashSet<Node>();
        Node init = initialNode();
        nodesToOpen.add(init);
        knownNodes.add(init);

        Node answer = null;
        while (!nodesToOpen.isEmpty())
        {
            Node n = nodesToOpen.poll();
            if (n.satisfiedCnt == customers.length) {
                answer = n;
                break;
            }
            for (int custIdx=0; custIdx< customers.length; custIdx++)
            {
                if (n.satisfiedStates[custIdx]) continue;
                List<Node> newNodes = n.generateNewNodes(customers[custIdx]);
                for (Node newN: newNodes)
                {
                    if (!knownNodes.contains(newN))
                    {
                        nodesToOpen.add(newN);
                        knownNodes.add(newN);
                    }
                }
            }
        }
        return answer;
    }
    public static class Customer{
        final   int index;
        final int flavorCnt;
        final int[] likedFlavors;
        final boolean[] likedMalted;

        public Customer(int custIdx, int custFlavorCnt, int[] likedFlavors, boolean[] likeMalted) {

            this.index = custIdx;
            this.flavorCnt= custFlavorCnt;
            this.likedFlavors = likedFlavors;
            this.likedMalted = likeMalted;
        }
    }

    public Node initialNode()
    {
        StringBuilder sb = new StringBuilder();
        for (int i=0; i<flavorCnt; i++ )
            sb.append('0');
        return new Node(sb.toString().toCharArray());
    }

    static char[] changeState(char[] origState, int flavorNo, boolean likeMalted)
    {
        char[] istate = origState.clone();
        if (likeMalted) {
            istate[flavorNo] = '1';
        }  else istate[flavorNo] = '0';
        return istate;
    }
    public class Node implements Comparable<Node> {
        final char[] state;
        final int satisfiedCnt;
        final int maltCost;
        final boolean[] satisfiedStates;
        List<Node> generateNewNodes(Customer c)
        {
            List<Node> r = new ArrayList<Node>();
            if (satisfiedStates[c.index]) return r;
            for (int flavorIdx=0; flavorIdx<c.flavorCnt; flavorIdx++)
            {
                int flavorNo = c.likedFlavors[flavorIdx];
                boolean likeMalted = c.likedMalted[flavorIdx];
                char currentFlavorState = state[flavorNo];
                // don't care current state
                char[] newstate = changeState(this.state,flavorNo,likeMalted);
                Node newNode = new Node(newstate);
                r.add(newNode);

            }
            return r;
        }

        public Node(char[] nstate)
        {
            state = nstate;
            int cntSatisfied=0;
            int cost =0;
            boolean[] satisfiedStates = new boolean[customers.length];
            for (int j=0; j<flavorCnt; j++)
            {
                if (nstate[j]=='1') cost++;
            }
            maltCost = cost;
            for (Customer c : customers)
            {
                boolean thisSatisfied = false;
                for (int i=0; i<c.flavorCnt; i++)
                {
                    int flavorNo = c.likedFlavors[i];
                    boolean likeMalted = c.likedMalted[i];
                    if (likeMalted)
                    {
                        if (state[flavorNo]== '1') thisSatisfied = true;
                    } else {
                        if (state[flavorNo]== '0') thisSatisfied = true;
                    }
                }
                if (thisSatisfied) {
                    cntSatisfied ++;
                    satisfiedStates[c.index] =true;
                } else satisfiedStates[c.index] = false;
            }
            satisfiedCnt = cntSatisfied;
            this.satisfiedStates =satisfiedStates;
        }

        @Override
        public int compareTo(Node o) {
            if (this.maltCost < o.maltCost)
                return -1;
            if (this.maltCost > o.maltCost)
                return 1;
            if (this.satisfiedCnt > o.satisfiedCnt)
                return -1;
            if (this.satisfiedCnt < o.satisfiedCnt)
                return 1;
            return 0;

        }
        @Override
        public boolean equals(Object o)
        {
            if (o instanceof Node)
            { Node oth = (Node)o;
               if (Arrays.equals(oth.state,this.state)) return true;
            }
            return false;
        }
        @Override
        public int hashCode()
        {
            return Arrays.hashCode(this.state);
        }


    }
}

Now the small problem set is solved, but run out of memory for the large problem set. The third solution changes the dijkstra algorithm again, I even renamed it because I weren't sure it is still dijkstra. Essentially it mixes customer iteration sorted by possibilites (the lesser possibilities iterated first) with dijkstra-style node opening. And only one customer is chosen to be opened in each iteration. The benefits are zero possibility cases are identified as dead ends and not opened further. Here are the sources, these code solved the large problem set :
package yudhi.pkg;

/**
 * Created with IntelliJ IDEA.
 * User: yudhi
 * Date: 3/15/14
 * Time: 5:59 PM
 * To change this template use File | Settings | File Templates.
 */


import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
import java.util.*;

/**
 * Created with IntelliJ IDEA.
 * User: yudhi
 * Date: 3/15/14
 * Time: 4:00 PM
 * To change this template use File | Settings | File Templates.
 */
public class MilkshakeC {
    public MilkshakeC(int flavorCnt, Customer[] customerList) {

        this.flavorCnt = flavorCnt;
        this.customers = customerList;
    }

    public static void main(String[] args) throws IOException {
        BufferedReader br = new BufferedReader(new InputStreamReader(System.in));
        String testCasesStr= br.readLine();
        int testCases = Integer.parseInt(testCasesStr);
        for (int caseNo=1; caseNo<=testCases; caseNo++)
        {
            String flavorCntStr = br.readLine();
            int flavorCnt = Integer.parseInt(flavorCntStr);
            int custCount = Integer.parseInt(br.readLine());
            Customer[] customerList = new Customer[custCount];
            for (int custIdx=0; custIdx<custCount; custIdx++)
            {
                String custspec = br.readLine();
                String[] words = custspec.split(" ");
                int custFlavorCnt = Integer.parseInt(words[0]);
                int[] likedFlavors = new int[custFlavorCnt];
                boolean[] likeMalted = new boolean[custFlavorCnt];
                for (int i=0; i<custFlavorCnt; i++)
                {
                    likedFlavors[i]= Integer.parseInt(words[i*2+1]);
                    likeMalted[i] = (Integer.parseInt(words[i*2+2]))>0;

                }
                customerList[custIdx] = new Customer(custIdx,custFlavorCnt,likedFlavors,likeMalted);
            }
            MilkshakeC a = new MilkshakeC(flavorCnt+1,customerList); // hack because our flavors start from 0
            Node result = a.doSomething();
            if (result == null)
            {
                System.out.println("Case #" + caseNo+": IMPOSSIBLE");
            }
            else
            {
                System.out.print("Case #" + caseNo+":");
                for (int j=1; j<= flavorCnt; j++) // start from 1
                {
                    System.out.print(' ');
                    if (result.state[j]=='*') System.out.print('0'); else
                        System.out.print(result.state[j]);
                }
                System.out.println();

            }

        }

    }
    final Customer[] customers;
    final int flavorCnt;
    public Node doSomething()
    {
        PriorityQueue<Node> nodesToOpen = new PriorityQueue<Node>();
        HashSet<Node> knownNodes = new HashSet<Node>();
        final Node init = initialNode();
        nodesToOpen.add(init);
        knownNodes.add(init);
        Node answer = null;
        while (!nodesToOpen.isEmpty())
        {
            final Node n = nodesToOpen.poll();
            if (n.satisfiedCnt == customers.length) {
                answer = n;
                break;
            }
            ArrayList<Customer> customersList = new ArrayList<Customer>();
            customersList.addAll(Arrays.asList(this.customers));
            Collections.sort(customersList, new Comparator<Customer>() {
                @Override
                public int compare(Customer o1, Customer o2) {
                    return o1.possibilities(n) - o2.possibilities(n);  //To change body of implemented methods use File | Settings | File Templates.
                }
            });
            Customer chosenCustomer = null;
            for (Customer currentCustomer : customersList )
            {
                int custIdx = currentCustomer.index;
                if (n.satisfiedStates[custIdx]) continue;
                if (currentCustomer.possibilities(n)==0) {
                    break;
                }
                chosenCustomer = currentCustomer; break;
            }
            if (chosenCustomer != null)
            {
                List<Node> newNodes = n.generateNewNodes(chosenCustomer);
                for (Node newN: newNodes)
                {
                    if (!knownNodes.contains(newN))
                    {
                        nodesToOpen.add(newN);
                        knownNodes.add(newN);
                    }
                }
            }
        }
        return answer;
    }
    public static class Customer{
        final   int index;
        final int flavorCnt;
        final int[] likedFlavors;
        final boolean[] likedMalted;
        public int possibilities(Node n)
        {
            int cnt = 0;      // 0 means impossible
            for (int i=0; i<flavorCnt;i++)
            {
                int flavorNo = likedFlavors[i];
                if (n.state[flavorNo] == '*') cnt++;
                if ((n.state[flavorNo] == '1') && likedMalted[i]) return 1; // already satisfied
                if ((n.state[flavorNo] == '0') && (!likedMalted[i])) return 1; // already satisfied

            }
            return cnt;
        }
        public Customer(int custIdx, int custFlavorCnt, int[] likedFlavors, boolean[] likeMalted) {

            this.index = custIdx;
            this.flavorCnt= custFlavorCnt;
            this.likedFlavors = likedFlavors;
            this.likedMalted = likeMalted;
        }
    }

    public Node initialNode()
    {
        StringBuilder sb = new StringBuilder();
        for (int i=0; i<flavorCnt; i++ )
            sb.append('*');
        return new Node(sb.toString().toCharArray());
    }

    static char[] changeState(char[] origState, int flavorNo, boolean likeMalted)
    {
        char[] istate = origState.clone();
        if (likeMalted) {
            istate[flavorNo] = '1';
        }  else istate[flavorNo] = '0';
        return istate;
    }
    public class Node implements Comparable<Node> {
        final char[] state;
        final int satisfiedCnt;
        final int maltCost;
        final boolean[] satisfiedStates;
        List<Node> generateNewNodes(Customer c)
        {
            List<Node> r = new ArrayList<Node>();
            if (satisfiedStates[c.index]) return r;
            for (int flavorIdx=0; flavorIdx<c.flavorCnt; flavorIdx++)
            {
                int flavorNo = c.likedFlavors[flavorIdx];
                boolean likeMalted = c.likedMalted[flavorIdx];
                char currentFlavorState = state[flavorNo];
                // don't care current state
                char[] newstate = changeState(this.state,flavorNo,likeMalted);
                Node newNode = new Node(newstate);
                r.add(newNode);

            }
            return r;
        }

        public Node(char[] nstate)
        {
            state = nstate;
            int cntSatisfied=0;
            int cost =0;
            boolean[] satisfiedStates = new boolean[customers.length];
            for (int j=0; j<flavorCnt; j++)
            {
                if (nstate[j]=='1') cost++;
            }
            maltCost = cost;
            for (Customer c : customers)
            {
                boolean thisSatisfied = false;
                for (int i=0; i<c.flavorCnt; i++)
                {
                    int flavorNo = c.likedFlavors[i];
                    boolean likeMalted = c.likedMalted[i];
                    if (likeMalted)
                    {
                        if (state[flavorNo]== '1') thisSatisfied = true;
                    } else {
                        if (state[flavorNo]== '0') thisSatisfied = true;
                    }
                }
                if (thisSatisfied) {
                    cntSatisfied ++;
                    satisfiedStates[c.index] =true;
                } else satisfiedStates[c.index] = false;
            }
            satisfiedCnt = cntSatisfied;
            this.satisfiedStates =satisfiedStates;
        }

        @Override
        public int compareTo(Node o) {
            if (this.maltCost < o.maltCost)
                return -1;
            if (this.maltCost > o.maltCost)
                return 1;
            if (this.satisfiedCnt > o.satisfiedCnt)
                return -1;
            if (this.satisfiedCnt < o.satisfiedCnt)
                return 1;
            return 0;

        }
        @Override
        public boolean equals(Object o)
        {
            if (o instanceof Node)
            { Node oth = (Node)o;
                if (Arrays.equals(oth.state, this.state)) return true;
            }
            return false;
        }
        @Override
        public int hashCode()
        {
            return Arrays.hashCode(this.state);
        }


    }
}

Sunday, March 9, 2014

Invoking SAP ABAP Web Service

Sometimes we need to call function modules in SAP using Web Service techniques instead of plain RFC. For example, using PHP platform, we might want to avoid installing saprfc extension, and in .NET platform, we might want to avoid NetCo (SAP .NET Connector). It seems that calling SAP Web Services have several pitfalls. This post would inform you, the reader, so you would not make the same mistake as I did.

Ensure the Web Service is configured with authentication

At minimum, the web service must be invoked with SAP user id  and password. This means in two sides : the SAP web service must be prepared to receive SAP user id & password, and the calling client (.NET framework, for example) must send appropriate authentication headers (in my case, Basic authentication). For the first part, set the Authentication Method/ Transport Channel Authentication in transaction SOAMANAGER to require username/password. See http://scn.sap.com/docs/DOC-38805 for a screenshot. Failing to do so might make the Web Service provider asks for no authentication headers, so the calling client would not send any headers. This would result in 'SRT Authorization Denied' errors.

Ensure the calling client doesn't do Keep Alives

The problem is, SAP Netweaver's Web Service providers doesn't like keep alives. The symptoms is these error message : 'The request was aborted' or 'The request was cancelled'.  When I am watching network packets, the symptoms are FIN-marked packet from the server just after the client sends the POST request with the proper authentication headers.
What actually happens is :
  • client sends POST request without authentication headers, with Keep Alive on.
  • server replies with Not Authorized Http header.
  • client repeats POST request,  now with authentication headers
  • server sends FIN, because server doesn't feel like communicating anymore after sending Not Authorized reply. (I am not a server/network expert, but why on earth the server didn't send FIN after sending Not Authorized?)
For clients such as .NET Web Service client, this is would stop your client from calling SAP Web Service, so the workaround is to disable keep alive.  You might need to copy Markus Reich's solution from here.
public partial class xyzService : System.Web.Services.Protocols.SoapHttpClientProtocol {

...

    protected override System.Net.WebRequest GetWebRequest(Uri uri)
    {
        System.Net.HttpWebRequest request = (System.Net.HttpWebRequest)base.GetWebRequest(uri);
        request.KeepAlive = false;
        request.ProtocolVersion = System.Net.HttpVersion.Version10;
        return request;
    }

...

}

Alternative: Force client to send authentication headers at first request

This is not standards-oriented, but this solution might be useful too : Force the web service client to send authentication headers at first request. So there is no need for a second client request.

protected override System.Net.WebRequest
GetWebRequest(Uri uri)
        {
            System.Net.HttpWebRequest request =
            (System.Net.HttpWebRequest)base.GetWebRequest(uri);
            if (this.PreAuthenticate)
            {
                System.Net.NetworkCredential nc =
                this.Credentials.GetCredential(uri, "Basic");
                if (nc != null)
                {
                    byte[] credBuf =
                    new System.Text.UTF8Encoding().
                    GetBytes(nc.UserName + ":" + nc.Password);
                    request.Headers["Authorization"] =
                    "Basic " + Convert.ToBase64String(credBuf);
                }
            }
            return request;
        }

Ensure your SAP web service user has S_SERVICE and/or SAP_BC_WEBSERVICE_CONSUMER

From what I read, we need SAP_BC_WEBSERVICE_CONSUMER role in the SAP userid used during authentication above. SRT Authorization denied errors might occurred because of missing authorization role/objects. The reference states that we only need S_SERVICE with correct fields. Please try either solutions. 

Try different WSDL options when having SOAP errors

I only vaguely remember that in the past using certain WSDL options resulted in problems such as SOAP errors. I can't remember whether document/literal or rpc/encoded is the correct one, in my latest experience the service works with document/literals WSDL.

ABAP protocol stack requires full domain in the URL

It is not that fully qualified domain name required because virtual host issues, but the ABAP web protocol stack short dumps when we call a URL using hostname only. It seems that the ABAP web protocol stack specifically looks for dots (.) in the host portion of the URL, and when finding none it would stop further processing. So please give your server a Fully Qualified Domain Name and calls it using the FQDN. Or just create an /etc/hosts entry with fake FQDN in the client, and call the web service using the fake FQDN. The tricky part is that your WSDL would contain service URL, but you need to override that if the URL contains no dots (.), replacing it with fake FQDN service URL.
References: