Monday, July 29, 2013

Android Json file read instead of property file read


While I am building an Android application. I wanted to read a property file and wanted to act my app depends on the property file configuration. But, property file can't handle array unless I add dependency Apache Common Configuration. As I am building an mobile application, it should be compact. for the reason, I just used Android JSONObject to resolve this.

Create "country.json" file and locate it Android device "/" directory

{
    "country": [
        {
            "name": "germany",
            "code": "DE",
            "continent": "Europe",
            "eat": "sausage"
        },    
        {
            "name": "france",
            "code": "FR",
            "continent": "Europe",
            "eat": "croissant"
        },
        {
            "name": "korea",
            "code": "KR",
            "continent": "ASIA",
            "eat": "rice"
        },
        {
            "name": "japan",
            "code": "JP",
            "continent": "ASIA",
            "eat": "fish"
        }
    ]
}


Create 2 method and use "getCountryList()" on your need.
private List<Country> getCountryList() throws Exception {

 File dirSDCard = Environment.getExternalStorageDirectory();
 File yourFile = new File(dirSDCard, "country.json");
 InputStream jsonStream = new FileInputStream(yourFile);
 JSONObject jsonObject = new JSONObject(InputStreamToString(jsonStream));
 JSONArray jsonArray = jsonObject.getJSONArray("country");
 List<Country> countryList = new ArrayList<Country>();

 for (int i = 0; i < jsonArray.length(); i++) {

  JSONObject jsonCountry = jsonArray.getJSONObject(i);
  Country country = new Country();
  country.setName(jsoncountry.getString("name"));
  country.setCode(jsoncountry.getString("code"));
  country.setContinent(jsoncountry.getString("continent"));
  country.setEat(jsoncountry.getString("eat"));

  countryList.add(country);
 }

 return countryList;
}

private String InputStreamToString(InputStream is) {

 BufferedReader r = new BufferedReader(new InputStreamReader(is));
 StringBuilder total = new StringBuilder();
 String line;
 try {
  while ((line = r.readLine()) != null) {
   total.append(line);
  }
 } catch (IOException e) {
  e.printStackTrace();
 }
 return total.toString();

}

Monday, June 17, 2013

mybatis various datasource sample jndi, hsql, oracle, mysql, sqlite configuration setup with Springframework

While doing various projects, I had to use different DBs. I just summarized datasource for Mybatis samples which is after testing. You can simply comment out to for your prefer datasource.
If you like to know detail to implement this sample with Spring framework, please follow mybatis-integration-with-spring.

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
 xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:p="http://www.springframework.org/schema/p"
 xmlns:context="http://www.springframework.org/schema/context"
 xmlns:jdbc="http://www.springframework.org/schema/jdbc"
 xsi:schemaLocation="http://www.springframework.org/schema/beans
        http://www.springframework.org/schema/beans/spring-beans-3.0.xsd
        http://www.springframework.org/schema/context
        http://www.springframework.org/schema/context/spring-context-3.0.xsd
    http://www.springframework.org/schema/jdbc
        http://www.springframework.org/schema/jdbc/spring-jdbc-3.0.xsd">

<!-- JNDI datasource -->
<!-- 
<bean id="dataSource" class="org.springframework.jndi.JndiObjectFactoryBean">
 <property name="jndiName" value="jdbc/oraclePool" />
 <property name="resourceRef" value="true" />
</bean>
-->

<!-- Hsql datasource -->
<!-- 
<jdbc:embedded-database id="dataSource">
 <jdbc:script location="classpath:hsql/schema.sql" />
 <jdbc:script location="classpath:hsql/data.sql" />
</jdbc:embedded-database>
-->

<!-- Oracle JDBC datasource -->
<!-- 
<bean id="dataSource"
 class="org.springframework.jdbc.datasource.SimpleDriverDataSource">
 <property name="driverClass" value="oracle.jdbc.OracleDriver" />
 <property name="url" value="jdbc:oracle:thin:@HOST:1111:ORA" />
 <property name="username" value="ID" />
 <property name="password" value="PASSWORD" />
</bean>
-->

<!-- mysql JDBC datasource -->
<!-- 
<bean id="dataSource"
 class="org.springframework.jdbc.datasource.SimpleDriverDataSource">
 <property name="driverClass" value="com.mysql.jdbc.Driver" />
 <property name="url" value="jdbc:mysql://localhost:3306/mydata" />
 <property name="username" value="user123" />
 <property name="password" value="12345678" />
</bean>
 -->

<!-- sqlite JDBC datasource -->
<!-- 
<bean id="dataSource"
 class="org.springframework.jdbc.datasource.SimpleDriverDataSource">
 <property name="driverClass" value="org.sqlite.JDBC" />
 <property name="url" value="jdbc:sqlite:C:/yourdb.sqlite" />
</bean>
 -->

<bean id="sqlSessionFactory" class="org.mybatis.spring.SqlSessionFactoryBean">
 <property name="dataSource" ref="dataSource" />
 <property name="typeAliasesPackage" value="com.devtrigger.model" />
 <property name="mapperLocations" value="classpath*:dao/**/*.xml" />
</bean>

<bean id="sqlSession" class="org.mybatis.spring.SqlSessionTemplate">
 <constructor-arg index="0" ref="sqlSessionFactory" />
</bean>

<!-- scan for mapper interface files and let them be autowired -->
<bean class="org.mybatis.spring.mapper.MapperScannerConfigurer">
 <property name="basePackage" value="com.devtrigger.dao" />
</bean>

<bean id="transactionManager"
 class="org.springframework.jdbc.datasource.DataSourceTransactionManager">
 <property name="dataSource" ref="dataSource" />
</bean>

</beans>

Thursday, March 14, 2013

Useful SVN adminitrator command

Export svn
- When I don't know SVN credential. but I have access to repository server
$svn export --force file:///home/svn/myrepos ./myrepos


Check User access control
$cat /home/svn/myrepos/conf/authz
$cat /home/svn/myrepos/conf/passwd
$cat /home/svn/myrepos/conf/svnserve.conf


Create version 1.6 compatible Repository
$svnadmin create myrepos --pre-1.6-compatible


Dump svn repository
$svnadmin dump /home/svn/myrepos > /home/backup/svn/myrepos.dump


Incremental dump svn repository
- SVN incremental Backup
$ svnadmin dump myrepos --revision 0:1000 > dumpfile1
$ svnadmin dump myrepos --revision 1001:2000 --incremental > dumpfile2
$ svnadmin dump myrepos --revision 2001:3000 --incremental > dumpfile3
$ svnadmin dump <repos> -r 58:HEAD --deltify > <file2> 


Import dump file
$ cd /home/svn
$ svnadmin load --bypass-prop-validation myrepos < /home/backup/svn/myrepos.dump


Incremental import dump file
- SVN recovery 
* incremental :
$ svnadmin load < ~/repos-0-1000.svn_dump
$ svnadmin load < ~/repos-1000-2000.svn_dump
$ svnadmin load < ~/repos-2000-3000.svn_dump


Kill SVN daemon on Solaris
$ pkill -KILL svnserve


Start SVN daemon
$ svnserve -d -r /home/svn/


Pack svn
- SVN revision pack per 1000 revision
$ svnadmin pack /home/svn/sfc


Tuesday, December 11, 2012

Mybatis integration with Spring 3.1 Framework.

Hi All,

I have configured new web project with mybatis and springframework several times. due to several project and being asked from team member to set up another project springframework skeleton. There are many advantage to start up from scratch. then you will know what it all means. But, When you need a quick start up, it would be good to have a project template.

As I have configured it several times. I improved of understanding of these framework setting then I made simple optimized web project template for share with others.

This sample will show the result of Json String.

I recommend you to look through below source roughly and download attached sample source to test running and understanding. it will run fine on tomcat I have tested. you will not need any db connection for sample code. below sample is running based on memory DB Hsql.

Sample Source is attached.
you can download from below link
Download

To integrate mybatis with Spring framework I recommend you to follow below step :

1. create java web project.
2. get all dependencies and configure classpath ( I am going to use maven )
2. configure springframework.
3. configure mybatis
4. create classes and xml ( Mapper interface, Mapper xml, Spring controller, create beans)
5. run it on tomcat or any WAS
6. check result


Directory Structure, you can refer below



Dependency Overview :




Maven Dependency : pom.xml
Springframework 3.1.3, Jackson Mapper 1.9.11, Hsql 2.2.8
I have excluded mybatis spring dependency which depends on old verion of springframework

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
 xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
 <modelVersion>4.0.0</modelVersion>

 <groupId>com.devtrigger</groupId>
 <artifactId>web-template</artifactId>
 <version>1.0-SNAPSHOT</version>
 <packaging>war</packaging>

 <name>Web Template Project, you can copy this and use for new project set up quickly</name>

 <properties>
  <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
  <spring-version>3.1.3.RELEASE</spring-version>
 </properties>

 <dependencies>
  <dependency>
   <groupId>org.springframework</groupId>
   <artifactId>spring-orm</artifactId>
   <version>${spring-version}</version>
  </dependency>
  <dependency>
   <groupId>org.springframework</groupId>
   <artifactId>spring-webmvc-portlet</artifactId>
   <version>${spring-version}</version>
  </dependency>
  <dependency>
   <groupId>org.springframework</groupId>
   <artifactId>spring-oxm</artifactId>
   <version>${spring-version}</version>
  </dependency>
  <dependency>
   <groupId>org.springframework</groupId>
   <artifactId>spring-test</artifactId>
   <version>${spring-version}</version>
   <scope>test</scope>
  </dependency>
  <dependency>
   <groupId>org.mybatis</groupId>
   <artifactId>mybatis-spring</artifactId>
   <version>1.1.1</version>
   <exclusions>
    <exclusion>
     <groupId>org.springframework</groupId>
     <artifactId>spring-context</artifactId>
    </exclusion>
    <exclusion>
     <groupId>org.springframework</groupId>
     <artifactId>spring-core</artifactId>
    </exclusion>
    <exclusion>
     <groupId>org.springframework</groupId>
     <artifactId>spring-jdbc</artifactId>
    </exclusion>
    <exclusion>
     <groupId>org.springframework</groupId>
     <artifactId>spring-test</artifactId>
    </exclusion>
    <exclusion>
     <groupId>org.springframework</groupId>
     <artifactId>spring-tx</artifactId>
    </exclusion>
   </exclusions>
  </dependency>
  <dependency>
   <groupId>log4j</groupId>
   <artifactId>log4j</artifactId>
   <version>1.2.17</version>
  </dependency>
  <dependency>
   <groupId>org.codehaus.jackson</groupId>
   <artifactId>jackson-mapper-asl</artifactId>
   <version>1.9.11</version>
  </dependency>
  <dependency>
   <groupId>javax.servlet</groupId>
   <artifactId>jstl</artifactId>
   <version>1.2</version>
  </dependency>
  <dependency>
   <groupId>javax.servlet</groupId>
   <artifactId>servlet-api</artifactId>
   <version>2.5</version>
   <scope>provided</scope>
  </dependency>
  <dependency>
   <groupId>javax</groupId>
   <artifactId>javaee-web-api</artifactId>
   <version>6.0</version>
   <scope>provided</scope>
  </dependency>
  <dependency>
   <groupId>org.hsqldb</groupId>
   <artifactId>hsqldb</artifactId>
   <version>2.2.8</version>
  </dependency>
 </dependencies>

 <build>
  <plugins>
   <plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-compiler-plugin</artifactId>
    <version>2.3.2</version>
    <configuration>
     <source>1.6</source>
     <target>1.6</target>
     <encoding>utf-8</encoding>
    </configuration>
   </plugin>
   <plugin>
    <artifactId>maven-war-plugin</artifactId>
    <version>2.2</version>
   </plugin>
   <plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-eclipse-plugin</artifactId>
    <version>2.8</version>
    <configuration>
     <additionalProjectFacets>
      <jst.web>2.5</jst.web>
     </additionalProjectFacets>
     <downloadJavadocs>true</downloadJavadocs>
     <downloadSources>true</downloadSources>
     <wtpContextName>/</wtpContextName>
     <wtpdefaultserver>${eclipse.wtpdefaultserver}</wtpdefaultserver>
     <wtpversion>2.0</wtpversion>
    </configuration>
   </plugin>
  </plugins>
 </build>
</project>


Mybatis configuration : mybatis-context.xml
If you comment out or in, you can change datasource.
I have prepared sample for JDNI, JDBC and Hsql datasources which are all tested. Currently, It is running on Hsql for simple sample running on your memory based DB.
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
 xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:p="http://www.springframework.org/schema/p"
 xmlns:context="http://www.springframework.org/schema/context"
 xmlns:jdbc="http://www.springframework.org/schema/jdbc"
 xsi:schemaLocation="http://www.springframework.org/schema/beans
        http://www.springframework.org/schema/beans/spring-beans-3.0.xsd
        http://www.springframework.org/schema/context
        http://www.springframework.org/schema/context/spring-context-3.0.xsd
  http://www.springframework.org/schema/jdbc
        http://www.springframework.org/schema/jdbc/spring-jdbc-3.0.xsd">

 <!-- JNDI datasource -->
 <!-- 
 <bean id="dataSource" class="org.springframework.jndi.JndiObjectFactoryBean"> 
  <property name="jndiName" value="jdbc/oraclePool"/>
  <property name="resourceRef" value="true" />
 </bean> 
  -->
 
 <!-- Hsql datasource -->
 <!--
 -->
 <jdbc:embedded-database id="dataSource" >
  <jdbc:script location="classpath:hsql/schema.sql"/>
  <jdbc:script location="classpath:hsql/data.sql"/>
 </jdbc:embedded-database>

 <!-- JDBC datasource -->
 <!--
 <bean id="dataSource"
  class="org.springframework.jdbc.datasource.SimpleDriverDataSource">
  <property name="driverClass" value="oracle.jdbc.OracleDriver" />
  <property name="url" value="jdbc:oracle:thin:@HOST:1111:ORA" />
  <property name="username" value="ID" />
  <property name="password" value="PASSWORD" />
 </bean>
 -->

 <bean id="sqlSessionFactory" class="org.mybatis.spring.SqlSessionFactoryBean">
  <property name="dataSource" ref="dataSource" />
  <property name="typeAliasesPackage" value="com.devtrigger.model" />
  <property name="mapperLocations" value="classpath*:dao/**/*.xml" />
 </bean>

 <bean id="sqlSession" class="org.mybatis.spring.SqlSessionTemplate">
  <constructor-arg index="0" ref="sqlSessionFactory" />
 </bean>

 <!-- scan for mapper interface files and let them be autowired -->
 <bean class="org.mybatis.spring.mapper.MapperScannerConfigurer">
  <property name="basePackage" value="com.devtrigger.dao" />
 </bean>

 <bean id="transactionManager"
  class="org.springframework.jdbc.datasource.DataSourceTransactionManager">
  <property name="dataSource" ref="dataSource" />
 </bean>

</beans>



XML Mapper : JobMapper.xml
You need to write your actual sql queries. value will be replace by "#{property}"
"<property name="typeAliasesPackage" value="com.devtrigger.model" />" is configured on mybatis-context.xml to make it short to write of bean name. You would need to write "com.devtrigger.model.JobInfo" for using a java bean or need to be used typeAlias. Please aware that "jobInfo" is id of result map, "JobInfo" is a java bean.
<?xml version="1.0" encoding="UTF-8" ?>
<!DOCTYPE mapper
  PUBLIC "-//mybatis.org//DTD Mapper 3.0//EN"
  "http://mybatis.org/dtd/mybatis-3-mapper.dtd">
<mapper namespace="com.devtrigger.dao.JobMapper">

 <cache />

 <resultMap type="JobInfo" id="jobInfo">
  <result property="id" column="id" />
  <result property="panNumber" column="pan_number" />
  <result property="address" column="address" />
  <result property="city" column="city" />
  <result property="state" column="state" />
  <result property="pincode" column="pincode" />
 </resultMap>

 <select id="selectAll" resultMap="jobInfo">
  <![CDATA[
  select id, pan_number, address, city, state, pincode
  from JBT_MEM
  ]]>
 </select>

 <insert id="insert" parameterType="JobInfo">
  <![CDATA[
  insert into JBT_MEM values(#{id}, #{pan_number}, #{address}, #{city}, #{state}, #{pincode})
  ]]>  
 </insert>

 <update id="update" parameterType="JobInfo">
  <![CDATA[
  ]]> 
 </update>


 <delete id="delete" parameterType="JobInfo">
  <![CDATA[
  ]]> 
 </delete>
</mapper>


Mapper Interface : JobInfo.java
you need to have interface for mapping with Mapper xml file. the method name id must mapped with ID of SQL described in the mapper xml.
package com.devtrigger.dao;

import java.util.List;

import com.devtrigger.model.JobInfo;

public interface JobMapper {

 List<JobInfo> selectAll();
 int insert(JobInfo sampleInfo);
 int update(JobInfo sampleInfo);
 int delete(JobInfo sampleInfo);
}


Spring Controller : IndexController.java

package com.devtrigger.controller;

import java.util.List;

import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Controller;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.ResponseBody;

import com.devtrigger.model.JobInfo;
import com.devtrigger.service.JobService;

@Controller
public class IndexController {

 private final Log log = LogFactory.getLog(this.getClass());

 @Autowired
 private JobService jobService;
 
 @RequestMapping(value = "/index")
 @ResponseBody
 public List<JobInfo> requestIndex(){
  
  log.debug("controller is called");
  return jobService.getSampleInfo();
 }
 
}



type URL "http://localhost:8080/index.do" on your browser then you will see Json String returns. The reason I show Json result instead of jsp servlet result is, because you will find many sample on internet for jsp servlet result return
BTW, you need to put your jsp file under "/WEB-INF/jsp/" if you need.

[{"id":"1","panNumber":"AABBAABB","address":"Address","city":"NY","state":"AB","pincode":23500},{"id":"2","panNumber":"BGDBCBDB","address":"Address","city":"LA","state":"St","pincode":23500}]



Friday, November 23, 2012

Sample http/https connection with Apache HttpComponents in java

Hi All
While I am building common project for our team. I think it would be worth it to share with others. even it just started to build though.

I remember few times that url connection's destination server has been down for few hours and our service also effected due to the destination server down. because, we didn't set any time out for url connection. which consumed all of JVM thread and the result was, Stuck Thread occured, no thread from thread pool.

For building url connection. better set time out.

There are few things more consider to make a URL connection.
- stable, stable, stable
- in case of no response from destination server for long time, it should close connection
- http connection pooling
- https connection
- connection via proxy
- via credential access


The reason to make this source is, Apache httpclient is very powerful for making url connection, but without looking into detail, it is difficult to use right a way.



Source code structure :

Maven Dependency : pom.xml file
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
 xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
 <modelVersion>4.0.0</modelVersion>
 <groupId>com.devtrigger</groupId>
 <artifactId>sample</artifactId>
 <packaging>jar</packaging>
 <version>1.0-SNAPSHOT</version>
 <name>sample</name>
 <url>http://maven.apache.org</url>
 <dependencies>
  <dependency>
   <groupId>junit</groupId>
   <artifactId>junit</artifactId>
   <version>4.10</version>
   <scope>test</scope>
  </dependency>
  <dependency>
   <groupId>org.apache.httpcomponents</groupId>
   <artifactId>httpclient</artifactId>
   <version>4.2.2</version>
  </dependency>
  <dependency>
   <groupId>org.apache.commons</groupId>
   <artifactId>commons-lang3</artifactId>
   <version>3.1</version>
  </dependency>
  <dependency>
   <groupId>commons-configuration</groupId>
   <artifactId>commons-configuration</artifactId>
   <version>1.9</version>
  </dependency> 
  <dependency>
   <groupId>org.codehaus.jackson</groupId>
   <artifactId>jackson-mapper-asl</artifactId>
   <version>1.9.11</version>
  </dependency>   
 </dependencies>
    <build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
                <version>2.3.2</version>
                <configuration>
                    <source>1.6</source>
                    <target>1.6</target>
                    <encoding>utf-8</encoding>
                </configuration>
            </plugin>
   <plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-eclipse-plugin</artifactId>
    <version>2.8</version>
    <configuration>
     <downloadJavadocs>true</downloadJavadocs>
     <downloadSources>true</downloadSources>
    </configuration>
   </plugin>            
        </plugins>
    </build>  
</project>
Configuration : conf.xml
<?xml version="1.0" encoding="ISO-8859-1" ?>
<sample>
 <proxyServer>
  192.168.1.1
 </proxyServer>
 <proxyPort>
  8080
 </proxyPort>
</sample>
HttpClientService.java
package com.devtrigger;

import java.security.cert.CertificateException;
import java.security.cert.X509Certificate;

import org.apache.commons.configuration.ConfigurationException;
import org.apache.commons.configuration.XMLConfiguration;
import org.apache.commons.lang3.StringUtils;
import org.apache.http.HttpHost;
import org.apache.http.HttpVersion;
import org.apache.http.auth.AuthScope;
import org.apache.http.auth.UsernamePasswordCredentials;
import org.apache.http.conn.ClientConnectionManager;
import org.apache.http.conn.params.ConnRoutePNames;
import org.apache.http.conn.scheme.PlainSocketFactory;
import org.apache.http.conn.scheme.Scheme;
import org.apache.http.conn.scheme.SchemeRegistry;
import org.apache.http.conn.ssl.SSLSocketFactory;
import org.apache.http.conn.ssl.TrustStrategy;
import org.apache.http.impl.client.DefaultHttpClient;
import org.apache.http.impl.conn.PoolingClientConnectionManager;
import org.apache.http.params.BasicHttpParams;
import org.apache.http.params.HttpConnectionParams;
import org.apache.http.params.HttpParams;
import org.apache.http.params.HttpProtocolParams;

public class HttpClientService {

 private static XMLConfiguration config;

 private static final int CONNECTION_TIMEOUT = 10000;
 private static final int SOCKET_TIMEOUT = 60000;

 private static HttpClientService httpClientService = new HttpClientService(); 
 
 private HttpClientService() {
  try {
   config = new XMLConfiguration("config.xml");
  } catch (ConfigurationException e) {
   e.printStackTrace();
  }
 }
 
 public static HttpClientService getInstance(){
  return httpClientService;
 }
 
 /**
  * @return
  */
 DefaultHttpClient getHttpClient() {
  return getHttpClient(CONNECTION_TIMEOUT, SOCKET_TIMEOUT);
 }

 /**
  * @param domain
  * @param userName
  * @param password
  * @return
  */
 DefaultHttpClient getHttpClient(String domain, String userName, String password){
  
  DefaultHttpClient httpClient = getHttpClient();
  
  httpClient.getCredentialsProvider().setCredentials(
                new AuthScope(domain, 443),
                new UsernamePasswordCredentials(userName, password));
  
  return httpClient;
 }
 /**
  * @param connectionTimeOut
  * @param sockeTimeOut
  * @return
  */
 private DefaultHttpClient getHttpClient(int connectionTimeOut, int sockeTimeOut) {
  try {

   HttpParams httpParams = new BasicHttpParams();
   
   setParams(httpParams, connectionTimeOut, sockeTimeOut);
   
   SSLSocketFactory sf = new SSLSocketFactory(new TrustStrategy() {
    public boolean isTrusted(X509Certificate[] chain, String authType) throws CertificateException {
     return true;
    }
   }, SSLSocketFactory.ALLOW_ALL_HOSTNAME_VERIFIER);
   
   SchemeRegistry registry = new SchemeRegistry();
   registry.register(new Scheme("http", 80, PlainSocketFactory.getSocketFactory()));
   registry.register(new Scheme("https", 443, sf));

   ClientConnectionManager ccm = new PoolingClientConnectionManager(registry);
   
   return new DefaultHttpClient(ccm, httpParams);
  } catch (Exception e) {
   return new DefaultHttpClient();
  }
 } 
 
 /**
  * @param httpParams
  * @param connectionTimeOut
  * @param sockeTimeOut
  */
 private void setParams(HttpParams httpParams, int connectionTimeOut, int sockeTimeOut){
  
  setHttpProxy(httpParams);
  setHttpProtocolParams(httpParams);
  setHttpConnectionParams(httpParams, connectionTimeOut, sockeTimeOut);
  
  httpParams.setBooleanParameter("http.protocol.expect-continue", false);
 }
 
 /**
  * @param httpParams
  */
 private void setHttpProxy(HttpParams httpParams) {
  
  String proxyHost = config.getString("proxyServer");

  if (!StringUtils.isEmpty(proxyHost)) {
   int proxyPort = Integer.parseInt(config.getString("proxyPort"));
   HttpHost proxy = new HttpHost(proxyHost, proxyPort);
   httpParams.setParameter(ConnRoutePNames.DEFAULT_PROXY, proxy);
  }  
  
 }

 /**
  * @param httpParams
  */
 private void setHttpProtocolParams(HttpParams httpParams){
  HttpProtocolParams.setVersion(httpParams, HttpVersion.HTTP_1_1);
  HttpProtocolParams.setContentCharset(httpParams, "utf-8");
 }
 
 /**
  * @param httpParams
  * @param connectionTimeOut
  * @param sockeTimeOut
  */
 private void setHttpConnectionParams(HttpParams httpParams, int connectionTimeOut, int sockeTimeOut){
  HttpConnectionParams.setConnectionTimeout(httpParams, connectionTimeOut);
  HttpConnectionParams.setSoTimeout(httpParams, sockeTimeOut);
 }
  
}


HttpRequestService.java

package com.devtrigger;

import java.io.BufferedInputStream;
import java.io.BufferedOutputStream;
import java.io.File;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.UnsupportedEncodingException;
import java.net.URI;
import java.net.URISyntaxException;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;

import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;
import org.apache.http.Header;
import org.apache.http.HttpEntity;
import org.apache.http.HttpResponse;
import org.apache.http.HttpStatus;
import org.apache.http.NameValuePair;
import org.apache.http.ParseException;
import org.apache.http.client.ClientProtocolException;
import org.apache.http.client.HttpClient;
import org.apache.http.client.entity.UrlEncodedFormEntity;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.client.methods.HttpPost;
import org.apache.http.client.methods.HttpRequestBase;
import org.apache.http.entity.StringEntity;
import org.apache.http.message.BasicNameValuePair;
import org.apache.http.util.EntityUtils;
import org.codehaus.jackson.JsonGenerationException;
import org.codehaus.jackson.map.JsonMappingException;
import org.codehaus.jackson.map.ObjectMapper;

public class HttpRequestService {

 HttpClientService httpClientService = HttpClientService.getInstance();
 
 private final Log log = LogFactory.getLog(this.getClass());

 ObjectMapper objectMapper = new ObjectMapper();
 
 private boolean useCredential = false; 
 
 private String domain;
 
 private String userName;
 
 private String password;
 
 
 private boolean isUseCredential() {
  return useCredential;
 }

 private void setUseCredential(boolean useCredential) {
  this.useCredential = useCredential;
 }

 private String getDomain() {
  return domain;
 }

 private void setDomain(String domain) {
  this.domain = domain;
 }

 private String getUserName() {
  return userName;
 }

 private void setUserName(String userName) {
  this.userName = userName;
 }

 private String getPassword() {
  return password;
 }

 private void setPassword(String password) {
  this.password = password;
 }

 public void enableUseCredential(String domain, String userName, String password){
  
  setDomain(domain);
  setUserName(userName);
  setPassword(password);
  setUseCredential(true);
 }
 
 public String doGetRequestNgetResponseBody(String url) {
  return doGetRequestNgetResponseBody(url, new ArrayList
()); } /** * @param context * @param url * @param headerList * @return */ public String doGetRequestNgetResponseBody(String url, List
headerList) { URI uri = null; try { uri = new URI(url); } catch (URISyntaxException e) { e.printStackTrace(); } HttpGet get = new HttpGet(uri); for (Header header : headerList) { get.addHeader(header); } return getResponseBody(get); } /** * @param context * @param url * @param headerList * @return */ public HttpResponse doGetRequestNgetResponse(String url, List
headerList) { URI uri = null; try { uri = new URI(url); } catch (URISyntaxException e) { e.printStackTrace(); } HttpGet get = new HttpGet(uri); for (Header header : headerList) { get.addHeader(header); } return getResponse(get); } /** * @param context * @param url * @return */ public HttpResponse doGetRequestNgetResponse(String url) { URI uri = null; try { uri = new URI(url); } catch (URISyntaxException e) { e.printStackTrace(); } HttpGet get = new HttpGet(uri); return getResponse(get); } /** * @param context * @param url * @param params * @return */ public String doPostRequestNgetResponseBody(String url, List params) { HttpPost post = new HttpPost(url); UrlEncodedFormEntity ent = null; try { ent = new UrlEncodedFormEntity(params); } catch (UnsupportedEncodingException e) { e.printStackTrace(); } if (ent != null) post.setEntity(ent); return getResponseBody(post); } /** * @param context * @param url * @param params * @return */ public HttpResponse doPostRequestNgetResponse(String url, List params) { HttpPost post = new HttpPost(url); UrlEncodedFormEntity ent = null; try { ent = new UrlEncodedFormEntity(params); } catch (UnsupportedEncodingException e) { e.printStackTrace(); } if (ent != null) post.setEntity(ent); return getResponse(post); } /** * @param context * @param url * @param jsonParams * @return */ public String doPostRequestNgetResponseBody(String url, Object jsonObject) { HttpPost post = new HttpPost(url); StringEntity ent = null; try { log.debug(objectMapper.writeValueAsString(jsonObject)); ent = new StringEntity(objectMapper.writeValueAsString(jsonObject)); ent.setContentType("application/json"); } catch (JsonGenerationException e) { // TODO Auto-generated catch block e.printStackTrace(); } catch (JsonMappingException e) { // TODO Auto-generated catch block e.printStackTrace(); } catch (UnsupportedEncodingException e) { // TODO Auto-generated catch block e.printStackTrace(); } catch (IOException e) { // TODO Auto-generated catch block e.printStackTrace(); } if (ent != null) post.setEntity(ent); return getResponseBody(post); } /** * @param context * @param httpRequest * @return */ private String getResponseBody(HttpRequestBase httpRequest) { HttpResponse response = getResponse(httpRequest); return getResponseBody(response); } /** * It returns response body string * * @param response * @return */ public String getResponseBody(HttpResponse response) { HttpEntity resEntity = null; if (response != null) resEntity = response.getEntity(); if (resEntity != null) { try { return EntityUtils.toString(resEntity); } catch (ParseException e) { e.printStackTrace(); } catch (IOException e) { e.printStackTrace(); } } return ""; } public boolean saveResponseBodyStream(HttpResponse response, String filePath) { boolean result = false; HttpEntity resEntity = null; if (response != null) resEntity = response.getEntity(); if (resEntity != null) { BufferedInputStream bis; try { bis = new BufferedInputStream(resEntity.getContent()); BufferedOutputStream bos = new BufferedOutputStream(new FileOutputStream(new File(filePath))); int inByte; while ((inByte = bis.read()) != -1) bos.write(inByte); bis.close(); bos.close(); } catch (IllegalStateException e) { // TODO Auto-generated catch block e.printStackTrace(); } catch (IOException e) { // TODO Auto-generated catch block e.printStackTrace(); } } return result; } /** * @param context * @param url * @param headerList * @return */ public HashMap doGetRequestNgetResponseHeader(String url, List
headerList) { URI uri = null; try { uri = new URI(url); } catch (URISyntaxException e) { e.printStackTrace(); } HttpGet get = new HttpGet(uri); for (Header header : headerList) { get.addHeader(header); } return getResponseHeader(get); } /** * @param context * @param url * @param params * @return */ public HashMap doPostRequestNgetResponseHeader(String url, List params) { HttpPost post = new HttpPost(url); UrlEncodedFormEntity ent = null; try { ent = new UrlEncodedFormEntity(params); } catch (UnsupportedEncodingException e) { e.printStackTrace(); } if (ent != null) post.setEntity(ent); return getResponseHeader(post); } /** * @param context * @param httpRequest * @return */ private HashMap getResponseHeader(HttpRequestBase httpRequest) { HttpResponse response = getResponse(httpRequest); return getResponseHeader(response); } /** * @param response * @param headerName * @return */ public String getResponseHeaderValue(HttpResponse response, String headerName) { HashMap header = getResponseHeader(response); return (String) header.get(headerName); } /** * @param response * @return */ public HashMap getResponseHeader(HttpResponse response) { if (response != null) { Header[] headers = response.getAllHeaders(); return converHeaders2Map(headers); } else { return new HashMap(); } } /** * when response parameter is null, it will return 410 Gone status. * * @param response * @return */ public int getResponseCode(HttpResponse response) { if (response != null) { return response.getStatusLine().getStatusCode(); } else { return HttpStatus.SC_GONE; } } /** * @param headers * @return */ private HashMap converHeaders2Map(Header[] headers) { HashMap hashMap = new HashMap(); for (Header header : headers) { hashMap.put(header.getName(), header.getValue()); } return hashMap; } /** * @param context * @param httpRequest * @return */ public HttpResponse getResponse(HttpRequestBase httpRequest) { HttpClient httpClient = null; if(!isUseCredential()){ httpClient = httpClientService.getHttpClient(); }else{ httpClient = httpClientService.getHttpClient(getDomain(), getUserName(), getPassword()); setUseCredential(false); } HttpResponse response = null; try { response = httpClient.execute(httpRequest); } catch (ClientProtocolException e1) { e1.printStackTrace(); } catch (IOException e1) { e1.printStackTrace(); } return response; } }



HttpRequestTest.java for Junit test run.
package com.devtrigger;

import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;
import org.junit.Assert;
import org.junit.Test;

public class HttpRequestTest {

  private final Log logger = LogFactory.getLog(this.getClass());
 
 @Test
 public void testRequest(){
 
  HttpRequestService service = new HttpRequestService();
  String response = service.doGetRequestNgetResponseBody("http://www.google.com/");
  
  logger.debug("response : " + response);
  
  Assert.assertTrue(response.substring(0, 20).matches(".*(?i)(HtMl).*"));
  
 }
 
}


Test Success

In the end, you can just use like this on your source code.
HttpRequestService service = new HttpRequestService();
String response = service.doGetRequestNgetResponseBody("http://www.google.com/");
further more, you can do post request with header vales or json http body request so on. probably you would need to extend more method.

I hope it helps :) even though, it is not optimized and refactored.



@Add Dec 4th 2012
I have simply looped the test cases 1000 times to my local server.
It worked well and look into regarding to if it is threadsafe and concurrent. It is thread safe. I need to look into more, it's not easy for me to follow up this. you can get more detail info below
http://hc.apache.org/httpcomponents-client-ga/tutorial/html/connmgmt.html
"HttpClient assumes complete control over the process of connection initialization and termination as well as I/O operations on active connections. However various aspects of connection operations can be influenced using a number of parameters."

http://hc.apache.org/httpcomponents-client-ga/httpclient/apidocs/org/apache/http/impl/conn/PoolingClientConnectionManager.html
"PoolingConnectionManager maintains a maximum limit of connection on a per route basis and in total. Per default this implementation will create no more than than 2 concurrent connections per given route and no more 20 connections in total. For many real-world applications these limits may prove too constraining, especially if they use HTTP as a transport protocol for their services. Connection limits, however, can be adjusted using HTTP parameters."

Monday, November 5, 2012

How to use Maven Module, Example from Spring Batch


I have tried to use maven module for related project set up in once. but, I was not able to find good sample or answer for me. So I opened up well known open source project source files who uses maven. I found one from Spring Batch.

Benefit for using maven module :
I can manage related dependencies version number in once
Several projects can be setup once in your IDE.
Your IDE (ex: eclipse ) will look the module project which applies your latest changes between the projects.
My team member will get all these benefit. Also I can save time to help team member.

It is little long to follow, so, I will explain as simple as possible for easy understanding.

Directory Structure

/juno_workspace/SpringSource-spring-batch-326da17$ find ./ -name 'pom.xml'
./pom.xml
./spring-batch-parent/pom.xml
./spring-batch-core/pom.xml
./spring-batch-infrastructure/pom.xml


./pom.xml file


The point from this file is, "packaging" is set to "pom" and defines "modules"
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
 <modelVersion>4.0.0</modelVersion>
 <groupId>org.springframework.batch</groupId>
 <artifactId>spring-batch</artifactId>
 <name>Spring Batch</name>
 <description>Spring Batch provides tools for enterprise batch or bulk processing. It can be used to wire up jobs, and track
 their execution, or simply as an optimization for repetitive processing in a transactional environment. Spring Batch is part of the Spring Portfolio.</description>
 <version>2.2.0.BUILD-SNAPSHOT</version>
 <packaging>pom</packaging>
 <modules>
  <module>spring-batch-parent</module>
  <module>spring-batch-infrastructure</module>
  <module>spring-batch-core</module>
  <module>spring-batch-test</module>
 </modules>


./spring-batch-parent/pom.xml file


The point from this file is, "packaging" is set to "pom" and defines "dependencyManagement"
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
 xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
 <modelVersion>4.0.0</modelVersion>
 <groupId>org.springframework.batch</groupId>
 <artifactId>spring-batch-parent</artifactId>
 <version>2.2.0.BUILD-SNAPSHOT</version>
 <name>Spring Batch Parent</name>
 <description>Spring Batch parent project.  Defines dependencies and common configuration for the build process.</description>
 <url>http://static.springframework.org/spring-batch/${project.artifactId}</url>
 <packaging>pom</packaging>
...

 <dependencyManagement>
  <dependencies>
...
   <dependency>
    <groupId>org.aspectj</groupId>
    <artifactId>aspectjweaver</artifactId>
    <version>1.5.4</version>
   </dependency>
...



./spring-batch-core/pom.xml file


The point from this file is, "packaging" is set to "jar" and defines parent
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
 <modelVersion>4.0.0</modelVersion>
 <artifactId>spring-batch-infrastructure</artifactId>
 <packaging>jar</packaging>
 <name>Infrastructure</name>
 <description><![CDATA[The Spring Batch Infrastructure is a set of 
 low-level components, interfaces and tools for batch processing 
 applications and optimisations.]]>
 </description>
 <url>http://static.springframework.org/spring-batch/${project.artifactId}</url>
 <parent>
  <groupId>org.springframework.batch</groupId>
  <artifactId>spring-batch-parent</artifactId>
  <version>2.2.0.BUILD-SNAPSHOT</version>
  <relativePath>../spring-batch-parent</relativePath>
 </parent>
...
 <dependencies>
  <dependency>
   <groupId>org.aspectj</groupId>
   <artifactId>aspectjweaver</artifactId>
   <optional>true</optional>
  </dependency>
...


./spring-batch-infrastructure/pom.xml file


The point from this file is, "packaging" is set to "jar" and defines parent
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
 <modelVersion>4.0.0</modelVersion>
 <artifactId>spring-batch-core</artifactId>
 <packaging>jar</packaging>
 <name>Core</name>
 <description>Core domain for batch processing, expressing a domain of Jobs, Steps, Chunks, etc.</description>
 <url>http://static.springframework.org/spring-batch/${project.artifactId}</url>
 <parent>
  <groupId>org.springframework.batch</groupId>
  <artifactId>spring-batch-parent</artifactId>
  <version>2.2.0.BUILD-SNAPSHOT</version>
  <relativePath>../spring-batch-parent</relativePath>
 </parent>
...
 <dependencies>
  <dependency>
   <groupId>org.aspectj</groupId>
   <artifactId>aspectjweaver</artifactId>
   <optional>true</optional>
  </dependency>
...


Additionally,


You can see, there is no version number described for "aspectjweaver" on "./spring-batch-core/pom.xml" and "./spring-batch-infrastructure/pom.xml" file. because the version number is described on "./spring-batch-parent/pom.xml" file in "dependencyManagement". That you can manage version number in once. It will be great to manage version number like Junit or common libraries

you can run mvn eclipse:eclipse and import the project on your eclipse. I will post detail how to import project for the next.

Saturday, September 29, 2012

customized log4j.property location by adding system property

I put log4j.properties in classpath, but it is not located classpath root directory. I put all properties into properties directory to keep root classpath organized clean structure. Since I moved the log4j.properties file, log4j makes error when process start up. I had to put below line which add system property.
-Dlog4j.configuration="properties/log4j.properties" then it works well now.

LIB_HOME=/home01/batch/batch/common_lib
JAVA_HOME=/home02/bea10/jdk160_24

CLASSPATH=${LIB_HOME}/activation-1.1.jar
CLASSPATH=${CLASSPATH}:${LIB_HOME}/api.services-50.jar
CLASSPATH=${CLASSPATH}:${LIB_HOME}/axis-1.4.jar
CLASSPATH=${CLASSPATH}:${LIB_HOME}/axis-jaxrpc-1.4.jar
CLASSPATH=${CLASSPATH}:${LIB_HOME}/axis-saaj-1.4.jar
CLASSPATH=${CLASSPATH}:${LIB_HOME}/axis-wsdl4j-1.5.1.jar
CLASSPATH=${CLASSPATH}:${LIB_HOME}/commons-beanutils-1.8.3.jar
CLASSPATH=${CLASSPATH}:${LIB_HOME}/commons-digester-1.8.jar
CLASSPATH=${CLASSPATH}:${LIB_HOME}/commons-discovery-0.2.jar
CLASSPATH=${CLASSPATH}:${LIB_HOME}/commons-lang-2.6.jar
CLASSPATH=${CLASSPATH}:${LIB_HOME}/commons-logging-1.0.4.jar
CLASSPATH=${CLASSPATH}:${LIB_HOME}/commons-net-3.1.jar
CLASSPATH=${CLASSPATH}:${LIB_HOME}/commons-validator-1.4.0.jar
CLASSPATH=${CLASSPATH}:${LIB_HOME}/ibatis-sqlmap-2.3.4.726.jar
CLASSPATH=${CLASSPATH}:${LIB_HOME}/log4j-1.2.16.jar
CLASSPATH=${CLASSPATH}:${LIB_HOME}/mail-1.4.5.jar
CLASSPATH=${CLASSPATH}:${LIB_HOME}/server.service-1.0.jar
CLASSPATH=${CLASSPATH}:${LIB_HOME}/servlet-api-2.5.jar
CLASSPATH=${CLASSPATH}:${LIB_HOME}/ojdbc6.jar
CLASSPATH=${CLASSPATH}:${LIB_HOME}/uk_batch.jar

export CLASSPATH 
sdate=`date +'%Y%m%d'`
stime=`date +'%H%M%S'`

${JAVA_HOME}/bin/java -Dlog4j.configuration="properties/log4j.properties" -cp ${CLASSPATH} batch.Run