The Use of SOAP Action with WSF/PHP

In SOAP 1.1, the SOAP action is mentioned as a compulsory HTTP header. In practice it was used to identify the operation for a given request which we identify by the term ‘dispatching operations’.
As of SOAP 1.2, the SOAP action field became optional, and it was part of the content type header.

In WSF/PHP, you can use the “action” property of WSMessage to set the SOAP action in a request SOAP message.

$requestPayloadString = <<<XML
<ns1:echoString xmlns:ns1="http://wso2.org/wsfphp/samples">
   <text>Hello World!</text>
</ns1:echoString>
XML;

$client = new WSClient(array( "to" => "http://localhost/samples/echo_service.php" ));

$requestMessage = new WSMessage($requestPayloadString,
	    array("action" => "http://localhost/samples/echo_service/echoString"));

$responseMessage = $client->request($requestMessage);

printf("Response = %s <br>", htmlspecialchars($responseMessage->str));

WSF/PHP send SOAP 1.2 request by default. So the SOAP message HTTP headers for the above code would be like this,

POST /samples/echo_service.php HTTP/1.1
User-Agent: Axis2C/1.5.0
Content-Length: 223
Content-Type: application/soap+xml;charset=UTF-8;action="http://localhost/samples/echo_service/echoString"
Host: 127.0.0.1

Note that the SOAP  Action is sent in the Content-Type inside the optional action parameter.

If you set it to use SOAP 1.1 explicitly in the WSClient like this,

$client = new WSClient(array( "to" => "http://localhost/samples/echo_service.php",
                              "useSOAP" => 1.1 ));

It would send the SOAP Action as a separate HTTP header,

POST /samples/echo_service.php HTTP/1.1
User-Agent: Axis2C/1.5.0
SOAPAction: "http://localhost/samples/echo_service/echoString"
Content-Length: 225
Content-Type: text/xml;charset=UTF-8
Host: 127.0.0.1

If you are using WSF/PHP in the server side, you don’t need to worry about the SOAP version as it will support both SOAP 1.1 and SOAP 1.2 automatically.
Anyway you need to declared the actions to operations map, so WSF/PHP will be able to dispatch the SOAP request and direct the message to the correct operation.

function echoStringFunc($inMessage) {
    // logic of echoString operation
}

function echoIntFunc($inMessage) {
    // logic of echoInt operation
}

// we will take echoString and echoInt as tow operations
$operations = array("echoString" => "echoStringFunction",
                    "echoInt" => "echoIntFunction");

// soap action to operation map
$actions = array("http://localhost/samples/echo_service/echoString" => "echoString",
                 "http://localhost/samples/echo_service/echoInt" => "echoInt");

// creating the service with the operations and actions set
$service = new WSService(array("operations" => $operations,
                               "actions" => $actions));

$service->reply();

So whenever a SOAP request message hit this script, it will check the SOAPAction HTTP header if it is a SOAP 1.1 message, or the action parameter in the Content-Type HTTP header if it is 1.2, then try to find the function mapped with that action from the actions map.
Anyway if the SOAP action is not set in the request message WSF/PHP will try to dispatch the message with other methods like body based dispatching, WS-Addressing based dispatching which are transport protocol independent dispatching mechanisms.

Anyway since most of the cases SOAP is used on top of HTTP, it is really common SOAP messages are dispatched using the SOAP action which is no doubt a very easy and efficient way of dispatching.

Posted in php, SOA, Tutorial/Guide, web services, wsf/php, wso2 | Tagged , , , , , | 1 Comment

Sending Binary with Apache Axis2/C ADB

Axis2/C ADB is a C language binding to the XML schema. ADB object model represents an XML specific to a given schema in a WSDL. You can use the Axis2 codegen tool to generate ADB codes for your WSDL and use that to build and parse your XMLs. The idea is, if you use ADB to build and parse your xmls, it will be really easy to do that and you don’t need to know or understand anything about the schema or the wsdl.

Apache Axis2/C to can be used to send and receive binaries as MTOM, SWA or base64 encoded. But ADB generated code still support to send and receive base64 encoded binaries only. So if you use contract first approach  with Axis2/C (i.e start with the WSDL, then write the service based on that), you have to use base64-encoded (non-optimized) as the binary transferring method. Note that you can use the other methods like MTOM or SWA, if you write the code to build and parse the xmls from AXIOM which is a general model for XML like DOM.

Say you have an element with base64Binary type in your request XML. So the schema for that element would be,

<xs:complexType name="Person">
    <xs:sequence>
        <xs:element name="image" type="xs:base64Binary"/>
        ... <!-- some more elements -->
    </xs:sequence>
</xs:complexType>

After you code generated, you will get the adb_person.h and adb_person.c files with the following function prototypes and the implementations,

        /**
         * Constructor for creating adb_Person_t
         * @param env pointer to environment struct
         * @return newly created adb_Person_t object
         */
        adb_Person_t* AXIS2_CALL
        adb_Person_create(
            const axutil_env_t *env );

        /**
         * Free adb_Person_t object
         * @param  _Person adb_Person_t object to free
         * @param env pointer to environment struct
         * @return AXIS2_SUCCESS on success, else AXIS2_FAILURE
         */
        axis2_status_t AXIS2_CALL
        adb_Person_free (
            adb_Person_t* _Person,
            const axutil_env_t *env);

        /**
         * Getter for image.
         * @param  _Person adb_Person_t object
         * @param env pointer to environment struct
         * @return axutil_base64_binary_t*
         */
        axutil_base64_binary_t* AXIS2_CALL
        adb_Person_get_image(
            adb_Person_t* _Person,
            const axutil_env_t *env);

        /**
         * Setter for image.
         * @param  _Person adb_Person_t object
         * @param env pointer to environment struct
         * @param arg_image axutil_base64_binary_t*
         * @return AXIS2_SUCCESS on success, else AXIS2_FAILURE
         */
        axis2_status_t AXIS2_CALL
        adb_Person_set_image(
            adb_Person_t* _Person,
            const axutil_env_t *env,
            axutil_base64_binary_t*  arg_image);

So you can manipulate the person element in c language using the create, get, set and free function.

    /* here the env is the axutil_env_t* instance - the axis2/c environment */
    FILE *f = fopen("./images/person.png", "r+");
    int binary_count;
    /* binary read a function you may write to read the binary data to the
     variable binary and the count to the variable binary_count */
    unsigned char *binary = binary_read(f, env, &binary_count);
    axutil_base64_binary_t *base64 = axutil_base64_binary_create_with_plain_binary(
                                        env, binary, binary_count);

    adb_Person_t *person = adb_Person_create(env);
    adb_Person_set_image(person, env, base64);

You can set this adb person directly to your request or to a setter of another adb instance to complete the ADB tree. So this way you can send binaries (as base64 encoded) using the Axis2/C ADB generated code.

Posted in adb, axis2/c, codegen, SOA, tool, Tutorial/Guide, web services, wsdl2c, xml, xml schema | Tagged , , , , | Leave a comment

Data Services – Databases For SOA

Database plays a big role in any day-today application. It is a major component from accounting, web portal, CMS, SaaS applications, search engines to all enterprise applications. In traditional MVC(Model-View-Controller) applications we talk about the Model component which represent the database.. And in 3-tier architectural pattern it is the ‘Data layer’ which represent the database and provide data to the ‘Logic Layer’ as per request.

As SOA evolves, database is becoming just one part of the ‘Data Layer’ or the ‘Model’ component. There are many data sources, data providers which are mostly deployed as web services which you can ask for data as you do with traditional databases. This gives you the advantages of the use of web services like interoperability, security and reliability and other WS-* support. And it helps you to get rid of the headache of installing different drivers for different databases and most importantly it removes the tight binding of your application to the database. So with adaption of SOA, the ‘Data Layer’ has been replaced by the ‘Service Consuming Layer’.

The story of the ‘Data Provider’ also changed with the SOA adaption. The new data sources are designed with the SOA in mind. And the legacy systems are wrapped by a service layer to make it more easier to consume. We use the term ‘Data Services’ for the data sources deployed as web services.

There are many public data services available as ‘REST’ which is a lighter way of providing services. The other way is the use of WS-* features like WS-Security, WS-Reliable Messaging to deploy the data services which are mostly adopted by the enterprise.

Today there are many tools around, that helps you to develop data services from existing databases. WSO2 provides an open source data service framework that allows you to build data services from simple xml configuration files without the need of writing a single line of code. And WSF/PHP also packed with a data services library that allow you to write a simple PHP script to build a data service.

PHP is one of the favorite language to write database back-ended web applications. Specially considering the fact that there are thousands of servers powered by the LAMP (Linux  + Apache + MySQL +PHP) stack, PHP data service library would comes useful to build around a web service around these existing data sources and make them SOA-aware.

Posted in DataServices, php, REST, SOA, web services, wsf/php, wso2 | Tagged , , , | Leave a comment

Organic Chemistry – Reaction Simulator

Organic Reaction Simulator

It is a tool that simulates the organic reactions and automatically generate the IUPAC names for the organic compounds drawn by you. This covers most of the syllabus of the Organic Chemistry for G.C.E. A/L examination.

Features

  • Nice Canvas to draw your Organic compounds.
  • Generates IUPAC names for the drawn component as possible. (It generates IUPAC names for almost all the compounds which G.C.E A/L examination expect students to know)
  • You can simulate reaction for organic compounds with preferred inorganic/organic compounds and the selected conditions. (Currently supporting 168 reactions)
  • Extensibility of adding IUPAC naming rules without the need of recompiling the code.
  • Extensibility of adding custom reactions, again without compiling a single line of code.

Here is a screen shot of the main panel when ‘Aldole reaction’ is simulated for some simple reactants.

Aldole Reaction - Reaction Simulator

Aldole Reaction - Reaction Simulator

The Drawing Panel

The ‘Reaction Simulator’ app provides 2 views for the user. One is the main panel which shows and manages reactions which is shown in the above figure. The other view is the drawing panel. It will be not much different from your favorite drawing application.

Drawing Panel - Reaction Simulator

Drawing Panel - Reaction Simulator

Here you can select elements and bond types to draw your organic compound. Additionally It has tools to move or delete selected parts of your drawing.

The other thing you may have already noticed, you don’t need to complete all the bonds for a particular element. Rather if you keep one side of a single bond unconnected, it will be automatically connected to a ‘H'(Hydrogen) at the rendering, similarly for double bond the default connecting element would be ‘O'(Oxygen), and for triple bond it is ‘N’ (Nytrogen). And you don’t even need to bother about connecting bonds with an element, as if you connected only one bond to ‘C’ all the other 3 possible bonds will be considered as connected with ‘H’s.

Just check the drawing panel on your own and find its user friendliness. Now after you finish designing your compound, just press “RETURN”.

IUPAC Name Generator - Reaction Simulator

IUPAC Name Generator - Reaction Simulator

Yea it will nicely render our compound, and look at the bottom, it has generated the IUPAC name for the compound.

The Main Panel

This contain a canvas that you can add, edit and delete organic compounds (so you will be directed to the ‘Drawing Panel’), some sidebar panels to select Inorganic compounds and additional conditions requires for the reactions and a set of buttons to start reactions and manage the result set. At the end of the sidebar there is a ‘help’ button that directed you to the user guide.

Extending IUPAC Naming Rules

If you go to the “Data/IUPAC/” directory (you can view svn from here, http://svn.dimuthu.org/organic_chemistry/Organic_Reaction_Simulator/Data/IUPAC/) you can find there are set of .txt files (infact xmls) that defines the IUPAC rules.

If you open one of them (we will take al.txt), it would be something like this

<IUPAC basename="al" subname="oxo" level="6" nonumber="true" affectto="1">
	<bond type="2">
		<element type="7">
		</element>
	</bond>
	<bond type="1">
		<element type="1">
		</element>
	</bond>
</IUPAC>

It is the naming rules for aldehyde compounds. (H-C=O). First it identify the the aldehyde by checking double bond to ‘O’ (which is the element type ‘7’), and a single bond to ‘H’ (which is the element type ‘1’). If the element group found, it will give the name ‘al’ or ‘oxo’ depending on whether it defines the main group of elements of the compound or not.

If you take a look at all these rules, you can have a good idea what each of these syntax mean. And may be you can add your own rules, if you find something missing or incorrect there.

Extending the Reactions

The reaction rules are stored in the “Data/Reactions/” directory ( http://svn.dimuthu.org/organic_chemistry/Organic_Reaction_Simulator/Data/Reactions/).

I will take the first reaction we studied in the ‘Organic Chemistry’ Class.

CH4 + Cl2 + hv (dim light) —————-> CCl4 + H2

Here is the rule defining that reaction. (cl2hv.txt)

<reaction inorganics="Cl2" conditions="hv">
	<check>
		<bond type="1">
			<element type="H">
			</element>
		</bond>
	</check>

	<reordering activity="remain">
		<bond type="1">
			<check>
				<element type="H">
				</element>
			</check>
			<reordering activity="remain">
				<element type="H">
					<reordering activity="replace" to="Cl">
					</reordering>
				</element>
			</reordering>
		</bond>
	</reordering>
</reaction>

If you take a close look at, what it says is if there exist inorganic ‘Cl2’ and condition ‘hv’ (reactions inorganics=”Cl2″ conditions=”hv”), check for a ‘H’ (element type=”H”) elements that is connected to a ‘C’ element (which is the root of the XML) through a single bond (bond type=”1″), then don’t replace the ‘C’ (reordering activity=”remain”) and don’t replace the bond type (reordering activity=”remain”), just replace the ‘H’ with ‘Cl’ (reordering activity=”replace” to=”Cl”). It is so simple as that.

These rules are applied to all the C elements in reactants. Look at the following figure for this rule in application.

Alkane + Cl2 Reaction - Reaction Simulator

Alkane + Cl2 Reaction - Reaction Simulator

These reaction rules can be as complex as you want. Specially when two or more reactants are involved in a reaction, rules defining that reaction will be little complex. Look at how aldole reaction (which is applied in the first figure of this post) is written in aldole.txt.

There are all together 168 reactions currently defined in this way. If you found some reaction is missing, feel free to add a another rule file defining that reaction. And if you want to share that with others, just let me know :), so I can integrate it in to the distribution.

Download

This software application is not yet in a official release. I just thought use this post to do a pre release of the ‘Reaction Simulator’. You can download the windows binary of the pre release version (343KB) from http://downloads.dimuthu.org/bins/chemistry/reaction_simulator_pre_release/reaction_simulator_pre_release.zip

Source Code

SVN location for the source code of ‘Reaction Simulator’ is http://svn.dimuthu.org/organic_chemistry/Organic_Reaction_Simulator/

Known Limitations – Possible Improvements

  • Generation of IUPAC names and simulations of reactions are not supported for compounds which involves Benzene ring.
  • IUPAC names generation is not supported for cyclic compounds which are anyway not part of the G.C.E. A/L syllabus.
  • The set of elements, available in drawing compounds, set of inorganic compounds and conditions, available in reactions are fixed and cannot be extended without changing the code and recompiling.
  • You can’t start with an IUPAC name and derive the compound. Currently you always have to start with drawing the compound and then generate the IUPAC.
  • Only for windows!

Little Background

4 years ago, When I was a level 2 student in the University of Moratuwa, I participated to a competition for making educational software tools organized by C.S.E and N.I.E (National Institute of Education). I submitted a software that teaches Organic Chemistry. It consisted of interactive tutorials targeting local G.C.E A/L exams with exercises and revisions in both Sinhala(My Mother tongue) and English languages(not in Unicode though). I got the second price for that.

After the award I decided to improve my software application by adding an ‘Organic Reaction Simulator’. In fact in the vacation of 2 weeks for the New Year, I could complete it.

Although the N.I.E supposed to distribute the applications submitted to the competition throughout the country, it didn’t happened. So I decided to publish at least the ‘Reaction Simulator’ application which I actually didn’t submitted to the competition.

Technologies Used

This is completely written in C++. It hasn’t use MFC, because I thought MFC is too heavy for such a small application. I used a lightweight, small image library code taken with some custom changes from some windows game programming book.

This is using XML to load data about reactions and IUPAC names which I have described in details in the early part of the blog. It uses a small inbuilt xml parser (just one recursive function) to parse these xmls.

So it has no depenedencies to third party libraries, You can just chekcout the souce code from the svn, open the visual studio project and compile it (press ‘F7’).

Posted in algorithm, Organic Chemistry, personal, screenshots, xml | Tagged , , , | 11 Comments

Apache Axis2/C RESTful URL Mapping Algorithm

Few weeks back I wrote a blog post about Writing RESTful Services in C which explain the use of Axis2/C REST API. Basically when you provide a HTTP Method (GET, POST, PUT or DELETE) and a HTTP URL, it is matched with a given HTTP method and a URL pattern in order to identify the operation and extract out the request parameters. For the example mentioned in the above blog, we can summarize the URL mapping like this.

Operation HTTP Method URL Pattern Example Requests
getSubjects GET subjects GET subjects
getSubjectInfoPerName GET subjects/{name} GET subjects/maths
getStudnets GET students GET students
getStudnetsInfoPerName GET students/{name} GET students/john
getMarksPerSubjectPerStudent GET students/{student}/marks/{subject} GET students/john/marks/maths

You can watch an application with this URL mapping in live, written using WSF/PHP which in fact run Axis2/C algorithms underneath.

Last week I updated this REST mapping algorithm and started a discussion about the changes on Axis2/C Dev list. I thought it would be better explain the idea on by blog too.

What the early algorithm (before my changes) did was, it search each pattern in the order it was declared, and returns when a match is found. Sequential searching for a matching pattern can reduce the performance as the number of operations grows. So my solutions was to keep the url pattern in a multi level (recursive) structure and match the url from one level to another.

Here is the structure of the ‘c struct’. (defined in src/core/util/core_utils.c)

/* internal structure to keep the rest map in a multi level hash */
typedef struct {
    /* the structure will keep as many as following fields */

    /* if the mapped value is directly the operation */
    axis2_op_t *op_desc;

    /* if the mapped value is a constant, this keeps a hash map of
    possible constants => corrosponding map_internal structure */
    axutil_hash_t *consts_map;

    /* if the mapped value is a param, this keeps a hash map of
    possible param_values => corrosponding_map_internal structre */
    axutil_hash_t *params_map;

} axutil_core_utils_map_internal_t;

Here is how it will looks like when the above URL pattern set (shown in the above table) is kept inside this multi-level (recursive) structure.

svc->op_rest_map  (hash)
                |
            "GET:students" --------- axutil_core_utils_map_internal_t (instance)
                |                                            |
                |                                        op_desc (axis2_op_t* for "GET students" op)
                |                                            |
                |                                        consts_map (empty hash)
                |                                            |
                |                                        params_map (hash)
                |                                                         |
                |                                                      "{student_id}" ------------- axutil_core_utils_map_internal_t (instance)
                |                                                                                            |
                |                                                                                op_desc (axis2_op_t* for "GET students/{student_id}" op)
                |                                                                                            |
                |                                                                                parms_map (empty hash)
                |                                                                                            |
                |                                                                                 const_map (hash)
                |                                                                                            |
                |                                                                                        "marks" ------------------- axutil_core_utils_map_internal_t (instance)
                |                                                                                                                            |
                |                                                                                                                    op_desc (NULL)
                |                                                                                                                            |
                |                                                                                                                   consts_map (empty hash)
                |                                                                                                                            |
                |                                                                                                                   params_map (hash)
                |                                                                                                                            |
                |                                                                                                                      "{subject_id}" ----------- axutil_core_utils_map_internal_t (instance)
                |                                                                                                                                                                               |
                |                                                                                                                                       op_desc (axis2_op_t* for "GET students/{student_id}/marks/{subject_id}" op)
                |                                                                                                                                                                               |
                |                                                                                                                                                                 consts_map / params_map (Both NULL)
                |
            "GET:students" --------- axutil_core_utils_map_internal_t (instance)
                                                            |
                                                        op_desc (axis2_op_t* for "GET students" op)
                                                            |
                                                        consts_map (empty hash)
                                                            |
                                                        params_map (hash)
                                                            |
                                                      "{student_id}" ------------- axutil_core_utils_map_internal_t (instance)
                                                                                                          |
                                                                                  op_desc (axis2_op_t* for "GET students/{student_id}" op)
                                                                                                          |
                                                                                             consts_map / params_map (Both NULL)

This structure is build at the time the server initialize the services. (from the “axis2_svc_get_rest_op_list_with_method_and_location” function in src/core/description/svc.c)

As each request hit the service, the request HTTP method and the URL is matched (which we call ‘rest dispatching’) with the above structure using the following algorithm. (defined in the “axis2_rest_disp_find_op” function in src/core/engine/rest_disp.c). Note that here we are extracting out the user REST parameters as well, but it is not shown in here.

  1. The request URL is spitted in to URL components from ‘/’ character. Retrive the instance of axutil_core_utils_map_internal_t  from the svc->rest_map to the varaible ‘mapping_struct’.
  2. Check the existance of URL components, count(URL components) > 0.
  3. If it doesn’t exist any URL components, get the value of mapping_struct->op_desc where the mapping_struct is the current mapping instance of axutil_core_utils_map_internal_t. if the mapping_struct->op_desc is not NULL, we found the operation. If it is NULL just exit returning NULL.
  4. Else If some URL component(s) exist, check the most former URL component in the mapping_struct->const_map hash. If mapping_struct->const_map[‘former_url_component’] is not NULL, assign the mapping struct->const_map[‘former_url_component’] value to mapping_struct and follow the step 2 with the remaining URL components. (note that here hash[‘key’] syntax is used to take the value for the key from the hash ‘hash’. If that returns TRUE, we found the opeartion, if not countine to step5.
  5. if mapping_struct->const_map[‘former_url_component’] is NULL, match the former url component with each key (which is a URL component pattern) in mapping_struct->param_map hash. (We use the function  “axis2_core_utils_match_url_component_with_pattern” in src/core/util/core_utils.c to map URL component with the URL component pattern). If matching pattern found assign the mapping_struct->param_map[‘key’] to mapping struct and follow the step 2 with the remaining URL components. If that returns TRUE for some key it will be the matching operation.

Where as the earlier algorithm can be simplified to,

  1. Match the request URL with URL patterns of each operation. This will be like calling the function “axis2_core_utils_match_url_component_with_pattern” (mentioned in step5 of the above algorithm) for the complete URL rather than for a URL component
  2. If the pattern is matched, matching operation is the selected operation for the request.

I approximately calculated the time complexity of both of these algorithm.

Here is the time complexity of the later described algorithm.

Average time complexity of iterating ‘n’ number of operations n/2 = O(n)
Time complexity of matching pattern with a URL with the length ‘p’ (complexity of the ‘axis2_core_utils_match_url_component_with_pattern’ function) O(p^2)
Complete time complexity of the algorithm O(n*p^2)

Time complexity of the formerly described algorithm. (which is currently in the SVN).

Time Complexity of a Hash Search O(1)
Average Number of has searches required. This is the average number of levels in the tree of recursive structures drawn above long(n)/2 = O(log(n))
Time complexity of matching pattern with a URL component with the average length ‘d’, d < p (p = the length of the complete URL) O(d^2)
Number of time pattern matching is required = number of param components in the URL = k, k < p/d (p = the length of the url, d = average length of the URL component)/ k = O(k)
Complete time complexity of the algorithm O(log(n)*d^2*k)

Considering the facts, O(logn) < O(n),d < p and k < p/d we can safely conclude

O(long(n)*d^2*k) < O(n*p^2)  => The newer algorithm has better (low) time complexity.

However the time complexity is valid only it takes high values for the parameters. For low values  the actual time taken by the newer algorithm can have high values, considering the constant overhead of the recursions and the hash search. So in order to judge the performance of the algorithm, we have to run some test cases and measure the actual times. Possibly a task for the weekend 🙂

Posted in algorithm, axis2/c, REST, RESTful, SOA, web services | Tagged , , , , | Leave a comment

Is Down Economy a Problem To Impelement SOA? Come to Open Source..

As the slow down of the economy your company may not tend to invest large amount of dollars to implement SOA in to your enterprise. But with the open source products you can implement your SOA platform with the same high quality, but in amazingly low price (which you may spend for support and consultancy).

WSO2, a leading open souce SOA company highligts this idea in this article “Open source lets developers speed SOA development despite economic slowdown“.

Posted in SOA, web services, wso2 | Tagged , , , , | Leave a comment

WS-SecurityPolicy With PHP

WS-SecurityPolicy specification defines standards for defining security policies for your web service. WSF/PHP allows you to declare your security policies according to these standards.

You can take one of following approaches to associate policies to your web service or client.

  • PHP Array to represent your policies
  • Policy file compliant with WS-Security Policy.
  • Declaring policies inline with the WSDL.

Declaring Policies with a PHP Array

This is a WSF/PHP specific API to declare policies for a web service. You don’t need to learn WS-Security Policy to write policies with this approach. You can set whether you want to use encryption, signing or usernameToken in a PHP array and create a WSPolicy object using it.

// here is the security array to declare your policies in simple manner
$sec_array = array("encrypt" => TRUE,
 "algorithmSuite" => "Basic256Rsa15",
 "securityTokenReference" => "IssuerSerial");

// creating WSPolicy instance using the policy array
$policy = new WSPolicy(array("security"=> $sec_array));

You can use this policy object to create a service along with a WSSecurityToken which contain the user parameters like the server private key and the client certificate.

$sec_token = new WSSecurityToken(array(
 "privateKey" => $server_pvt_key,
 "receiverCertificate" => $client_pub_key));

$svr = new WSService(array("actions" => $actions,
 "operations" => $operations,
 "policy" => $policy,
 "securityToken" => $sec_token)); // here is the policy object you just created

$svr->reply();

You can invoke this service just by writing a simple web service client. There also you need to provide the policies declared in the service, so the client can build his request to validate with server policies. You will be using a similar WSPolicy object to set these policies at the client side too, as show in the below code segment.

 $sec_token = new WSSecurityToken(array(
   "privateKey" => $pvt_key,
   "receiverCertificate" => $rec_cert));

 $client = new WSClient(array("useWSA" => TRUE,
    "policy" => $policy, /* the policy object */
    "securityToken" => $sec_token));

 $resMessage = $client->request($reqMessage);

Declaring Policies with a Policy File

You can set your policies in the server or client side using a policy file compliant with WS-Security Policy specification. You have to take this approach if your policy requirements are too complicated, like you want to sign only some parts of the message or you want to encrypt some soap headers.

Similar to the above method, here too you will use the WSPolicy object to set your policies. But unlike the above where you give the policies as a PHP array , here you can just give the policy file as an argument to the WSPolicy constructor.

// creating the WSPolicy instance from a policy file
$policy_xml = file_get_contents("policy.xml");
$policy = new WSPolicy($policy_xml);

Here is an example of a complete policy file written according to the WS-Security Policy standards. And you can find a quick guide on WS-Security Policy from the article Understanding WS-Security Policy Language written by Nandana, a key leader of Apache Rampart project.

Declaring Policies inline in a WSDL

We use WSDL to describe our web services. WSDL has the information about the service endpoint, the transport protocols (e.g. http), messaging protocols (e.g. SOAP) and the message schemas and many others about the service. You can attach your policies inside a WSDL.

Here is an example of a WSDL with inline policies. The difference in this approach is you can set your policies separately for each messages or each operations or each endpoints of your service. The following segment of a WSDL shows how you refer to different policies which are declared in the early part of the WSDL.

     <wsdl:binding name="CalendarSOAP12Binding" type="ns1:CalendarPortType">
       <!-- Endpoint policies are declared here.
          these are common to all messages transferring
          through this protocols (i.e. SOAP12, http)-->
        <wsp:PolicyReference URI="#transport_binding_policy"/>
        <soap12:binding transport="http://schemas.xmlsoap.org/soap/http" style="document"/>
        <wsdl:operation name="login">
           <soap12:operation soapAction="urn:login" style="document"/>
           <wsdl:input>
              <!-- policy specific to the 'login' operation -->
              <wsp:PolicyReference URI="#username_token_policy"/>
              <soap12:body use="literal"/>
           </wsdl:input>
           <wsdl:output>
              <soap12:body use="literal"/>
           </wsdl:output>
         </wsdl:operation>
         <wsdl:operation name="register">
            <!-- no specific policies are set for the 'register' operation
            <soap12:operation soapAction="urn:register" style="document"/>
            <wsdl:input>
              <soap12:body use="literal"/>
            </wsdl:input>
            <wsdl:output>
               <soap12:body use="literal"/>
            </wsdl:output>
         </wsdl:operation>
           ....
       </wsdl:binding>

This is the binding section of a WSDL where we bind messaging protocol and transport protocols with a service endpoint. Here we have “login” and “register” operations. Note that we are referring to “transport_binding_policy” from the parent level of each operation elements. That means these policies are common to all the operation in that binding. And inside the “login” operation we are referring to “username_token_policy”, so in order to invoke this operation, you have to send username token headers. And “register” doesn’t require any operation specific policies allowing users to register without any prior authentications.

You can select any of the above mentioned approach to define policies of your web service or to invoke a web service that support WS-Policy. If your policy requirements are simple, it will be easy to use the array based approach. If your policy requirements are complex or you have a good understanding of WS-Policy and WS-Security Policy you can rely on the policy file based approach or defining policy inline with WSDL. And the former 2 methods will give you a nice separation of the logic code and security configurations. The selection is yours:)

Posted in DataServices, php, REST, security, Tutorial/Guide, web services, WSDL, wsf/php, wso2, xml, xml schema | Tagged , , , , , , | 14 Comments

Signing SOAP Headers In PHP Web Services

Non-Repudiation and Integrity are two main security issues addressed by signing a message. If you are writing a web service or a service consumer in PHP you can use the WSF/PHP toolkit to sign messages.

Here is how you can sign a SOAP request message.

// loading the keys
$my_cert = ws_get_cert_from_file("client_certificate.cert");
$my_key = ws_get_key_from_file("client_private_key.pem");
$rec_cert = ws_get_cert_from_file("server_certificate.cert");

// preparing the policy array
$sec_array = array("sign"=>TRUE,
                   "algorithmSuite" => "Basic256Rsa15",
                   "securityTokenReference" => "IssuerSerial");

$policy = new WSPolicy(array("security"=>$sec_array));

// preparing the security_token
$sec_token = new WSSecurityToken(array("privateKey" => $my_key,
                                       "certificate" => $my_cert,
                                       "receiverCertificate" => $rec_cert));

// create the client using the policy and sec token + option ws-addressing
$client = new WSClient(array("useWSA" => TRUE,
                              "policy" => $policy,
                              "securityToken" => $sec_token));

// invoke the request				
$resMessage = $client->request($reqMessage);

Although I’ve not shown in the above code, you should always wrap the requesting code inside a try/catch block specially when there are security tokens in your message. Because whenever security test is failed the server will send a SOAP fault, and the client is expected to handle that.

Similarity you can write a service that accept only signing messages with the same policy and security tokens. But this time your security token contains the private key and the certificate of the server. And for service you create WSService instance (similarity to  WSClient) and feed these options.

// loading the keys and create the security token
$cert = ws_get_cert_from_file("server_certificate.cert");
$pvt_key = ws_get_key_from_file("server_private_key.pem")

$sec_token = new WSSecurityToken(array("privateKey" => $pvt_key,
                                       "certificate" => $cert));

// policy is declared similar to earlier example

$sec_array = array("sign" => TRUE,
                    "algorithmSuite" => "Basic256Rsa15",
                    "securityTokenReference" => "KeyIdentifier");

$policy = new WSPolicy(array("security"=>$sec_array));

// create a WSService instance and feed the options
$svr = new WSService(array("operations" => $operations,
                           "actions" => $actions,
                           "policy" => $policy,
                           "securityToken" => $sec_token));

$svr->reply();

With this code you will only be able to sign the payload of a SOAP request. Payload is what stays inside the soap body part which contain the actual data of your business. So in most of the case, what you want is integrity of your business data, so this will satisfy your need.

But there can be situation you want to sign SOAP headers which are outside the SOAP body. SOAP headers contain meta data of the message. For an example when you sign a message, the SOAP headers could carry the signature information and the certificates.

One example of the use of signing SOAP headers is replay detection. (I’ve written about replay detection on a early blog). in SOAP, replays can be detected using ‘message id’s which are transferred through the WS-addressing SOAP headers. But interceptors can change the message ids and send it as a unique message which will eventually cause to fail the replay detection. But if you sign the ‘message id’ (i.e. WS-Addressing headers) the intruders will not be able to do the trick since they can not recreate the signature for the changed content.

Anyway you can not set your requirement to sign the soap headers, just using a PHP array as it has done in the above examples. Rather you have to declare it using a policy file compliant to WS-SecurityPolicy specification. I.e instead of using a PHP array to create a WSPolicy instance like this,

$sec_array = array("sign"=>TRUE,
                    "algorithmSuite" => "Basic256Rsa15",
                    "securityTokenReference" => "IssuerSerial");

$policy = new WSPolicy(array("security"=>$sec_array));

You can create the WSPolicy instance directly using the policy file,

$policy_xml = file_get_contents("policy.xml");
$policy = new WSPolicy($policy_xml);

And inside the policy file, you can declare your requirements, policies. Here is how I say I want to sign all my WS-Addressing headers.

<sp:SignedParts xmlns:sp="http://schemas.xmlsoap.org/ws/2005/07/securitypolicy">
   <sp:Header Namespace="http://www.w3.org/2005/08/addressing"/>
</sp:SignedParts>

Note that here the namespace “http://www.w3.org/2005/08/addressing” is the namespace used in the Addressing headers. So I instruct to sign headers containing that namespace in order to sign the ‘message id’ inside the WS-Addressing headers. You can visit a comlete policy file with the declarations of signed part from here.

Posted in php, security, Tutorial/Guide, wsf/php, wso2 | Tagged , , , , , , , | Leave a comment

Detect Replay Attacks In to Your PHP Web Service

Replay attack is a common kind of attack, the hackers are using to break the security of a web service.

If you can intercept one soap message while it is transferring through the wire, you can replay that message to the server again and again. Since the original message may have already encrypted, signed and contain valid authentication credentials the replaying messages will also be able to pass all the security tests and fool the server and do enormous damages to the business. For an example think of replying a soap message that a client is conducting a payment with a e-commerce service. The service may tend to charge the client multiple times for each request which break the integrity of the business.

So it is no doubt when ever you thinking of designing an enterprise web service application, you should give attention to ‘Replay Detection’ more seriously.

One solution is you can handle the replay detection inside the business logic itself. If you do this for the above mentioned e-commerce like services, you will keep all the session ids and make sure only one payment is possible for one session. But this may need some really careful design of the application logic.

The other solution is to let your web service framework to handle the ‘Replay Detection’. That will clearly separate the security aspects from the business aspects of your service. And it will give you more flexibility in configuring your security requirements. And the other advantage is it will detect ‘Replay Attacks’ well before hitting the business logic, making the web service perform well.

WSF/PHP allows you to detect replay attacks using WS-Addressing and WS-Username token headers. WS-Addressing headers contains a message id which can be considered as unique to a soap message and ws-security headers contains created time of the message which can be used to calculate the age of the message and derive its validity.

WSF/PHP provide web service developer a callback with the ‘message id’ and the ‘message created time’  per each message. In the callback you can store this message id and created time in a database, and check them against all the incoming soap messages. If it found duplicate entries, you can consider it as a replay attack.

Here is an example draft of the above scenario written using WSF/PHP.

/* replay detection callback */
function replay_detect_callback($msg_id, $time_created) {
    /* Here is the pseudo code of the logic

    query for the $msg_id and $time_created for the database.
    if already exist
      return FALSE;
    else
      Insert message id and time created to the database
    return TRUE */
}

$security_token = new WSSecurityToken(array( 
                      "replayDetectionCallback" => "replay_detect_callback",
                      "enableReplayDetect" => TRUE,
                      /* Other tokens */ ));

$svr = new WSService(array("securityToken" => $security_token,
                      /* Other options*/ ));

$svr->reply()

So if you use this code in the service you can happily detect any replays of an old message and avoid possible damages. But remember this security check will work only if the replaying message contain the same message id and the created time as the original one. It is possible that an intruder not only intercept the message, but also change these fields and replay it in to your server. Such replays will not be detected from this code.

The solution is to sign each SOAP requests. If a client sign a message with his private key, the server can confirm that the message is not altered while it is on the wire. So if intruder replay a signed message, either it will fail the replay detection test (if it is replaying without changing the message id and created time) or it will fail validating the signature (if the message id and the created time is altered).

So if you implement a replay detection test with a signature test, you can eliminate all the replay attacks to your service (at least theoretically :).

Posted in php, security, Tutorial/Guide, web services, wsf/php, wso2 | Tagged , , , , , , | 2 Comments

RESTful URL Mapping in WSF/PHP

In a RESTful design, choose of URLs for resources are really important. The URL uniquely represents a resource. Service consumers can change some parts in the URL to access different other resources. So it is clear that the URL consists of some constant parts which describe the resource group or catalog in general and some variable parts which have different and unique values for different resources.

As an example look at the following URL patterns

  • students/{name} – The constant ‘students’ represent the students group in general and the variable ‘name’ is used to identify each student individually.
  • students/{name}/marks/{subject} – The constants ‘students’ and ‘marks’ shows that this resource is a marks of some students, The two variables ‘name’ and ‘subjects’ addresses which student and marks of which subject is presented in the URL.

You can have a look at some of the uses of such mappings from RESTful School demo.

WSF/PHP allows you to create RESTful Web Services and further more RESTful Data Services in PHP.

In a RESTful Data Service you expose a database query as a web service. There you can write a prepared statement and feed arguments for the statement through the variable parameters of the URL. For an example take the RESTfulSchool Demo Code.

To retrieve a particular student, we can use the following prepared statement and the URL pattern ( This URL Pattern+  HTTP ‘GET’ method is matched to execute this query. )

$sql = "SELECT * FROM Students where StudentName = ?"

$get_students_with_name_url = array("HTTPMethod" => "GET", 
                 "RESTLocation" => "students/{name}");

So you can execute this prepared statement with the subject name ‘John’ using the following URL.

http://labs.wso2.org/wsf/php/solutions/RESTFulSchool/school_service.php/students/john

If your service is not exposing the database directly, then you have to choose the general web service API rather than the data service specific API. In there you will be able to write your business logic for publishing student information in a PHP function and expose it as a web service.

In such a cas,e your function is taking an argument which of type WSMessage. This structure hold an XML that contains values for all the variable parameters as in the users request URL. For an example for above REST Mapping, we can expect the following XML.

<getSubject>
    <name>Chemistry</name>
</getSubject>

And your function to expose as a service, would be look like this,

function getSubject($input) {
    /* retrieve the subject name from the
       $input xml using simple xml */

    $input_xml = new SimpleXMLElement($input->str);

    $subject_name = $input->name;

    /* write the logic to retrieve subject information
      for the $subject_name */

    ....
}
Posted in DataServices, php, REST, RESTful, SQL, Tutorial/Guide, web services, wsf/php, wso2 | Tagged , , , , , , | 1 Comment