Monday, December 19, 2011

Productivity by Example: Liferay reverse-engineered

In this experiment, minuteproject demonstrates that it is possible to use Reverse-Engineering as a development methodology for Big Projects AND this from bootstrap time to the entire lifecycle of a Project.

To illustrate this postulat, I take enterprise-size opensource model (Liferay with150+ entities) upon which I apply minuteproject tracks.
It will quickly appear that performing 'bulk reverse-engineering' have strong limitations. Meanwhile by applying conventions deduced from the 'Reverse-Analysis' discipline, it will be possible to overcome them.
The generation will run releasing in couple of seconds (3 to 10) hundreds of artefacts.
Following Minuteproject philosophy, you'll will have to write 0 (Zero) LOC to get a working result and this can be achieved within 5 min-.
Eventually some code will be written on top and the result will be tested.
The technologies discussed will be:
  • JPA2 with Hibernate implementation
  • Querydsl
  • Ehcache 
  • Fitnesse
But other MP tracks can be applied.
The conclusion resumes the productivity benefit at project and enterprise level of embracing MinuteProject productivity philosophy.

Liferay
Liferay is an opensource portal, this article only focus on the database model.
Liferay Model
Setup 
The model is install on mysql database.
Now you can introspect your model.

Reverse-Analysis crash course
At first sight 150+ tables, 0 views.
Some tables start with QUARTZ, those are likely to be shipped with the Quartz scheduling framework, so they should be excluded from the generation.

At second sight (on mysql installation)
Entity with primary key but no foreign keys. => No relationships?
Mysql schema primary key are not autoincrement.

If reverse-engineering is performed as-is, then the bulk resulting artefacts in JPA2 will contain no relationships (no @OneToMany, @ManyToOne and @ManyToMany), so the resulting value will be quite poor (no graph navigation, query link to be done manually, no type safe benefit of Querydsl or Metamodel...)

Relation detection convention
Since the relationships are not formalised in terms of constraint, does it mean that there is none?
In fact, when having a deeper look inside the tables, some fields have name such as USER_ID and there is a strong chance that they are used for a relationship link with the User_ table primary key.
Let's assume it.
Since in minuteproject configuration the user can perform some enrichment and one of this enrichment consists in defining relationship, it is possible to enrich every table containing the field USER_ID by indicating it acts as a foreign key towards USER_ table.
// add snippet
It's fine, but it might be cumbersome since you'll have to do it more than 40 times and only for the 'towards User' relationships. What about the others?
A wiser way is to apply a convention:
What we need is to be able to detect relationships when those match a pattern.The pattern here is that the 'considered' foreign keys are fields ending with 'id' and whose beginning (column name without ending 'id') correspond to an existing table.
But this was not enough since some table ends with '_'.
So the convention has to take care of those exceptions and the match is done against the a map of those entity alias if the match was not successful via direct binding.
The final configuration of the convention is

<foreign-key-convention type="autodetect-foreign-key-based-on-similarity-and-map" 
column-ending="id" column-starting="" column-body="match-entity">
<property tag="map-entity" name="user_" value="user"/> 
<property tag="map-entity" name="group_" value="group"/>
<property tag="map-entity" name="organization_" value="organization"/>
<property tag="map-entity" name="permission_" value="permission"/>
</foreign-key-convention>

With this convention it is now possible to have 'virtual' foreign keys used by the generator to create @OneToMany, @ManyToOne and @ManyToMany relationships for JPA2 track.

Naming convention
The JEE conventions can be different from those retrieved from the Database structure:
Example: Some column ends with 'id' but in java we might not want that.
Here the convention for that
<column-naming-convention type="apply-strip-column-name-suffix" pattern-to-strip="ID" />

Collection naming convention
While reverse-engineering a problem is to give unambiguous names to list elements. Otherwise in java you have a compilation error. The safest way is to compose the name of this collection relationship (@OneToMany) with the name of the field holding the foreign key, and the name of entity having this FK. It get even worse when dealing with a many2many relationship where the intermediary link table as to be associated.
As a result, you can have very long name which is not quite handy.
Fortunately the convention apply-reference-alias-when-no-ambiguity is there for you!
<reference-naming-convention type="apply-referenced-alias-when-no-ambiguity" is-to-plurialize="true" />
It checks if while reducing the 'unambiguous' default name to 'just' the associated table (plurialize as option) there is no colision.
Remark
With conventions bear in mind that they are applied sequentially and so the order is very important!
Aliasing 
Entity name or column name can be aliased.
The alias will be used for the Java name but the ORM mapping will match the alias name to the correct entity name. 
Caching
To enable caching add a property to the target JPA2
               <property name="add-cache-implementation" value="ehcache"></property>
It is possible to associate caching to entities by two means:
By enrichment at entity level: Each entity marked as having their content-type="master-data" or ="reference-data" will have an associate cache entry
<entity name="country" content-type="reference-data"></entity>
By convention: describe a pattern of entity matching the requirement. Example all the entity ending with '_' could be considered as reference data
<entity-content-type-convention type="apply-content-type-to-entity-ending-with" pattern="_" content-type="reference-data"></entity-content-type-convention>
In this case the entity 'user', 'account', 'classname', 'contact', 'group', 'lock', 'organisation', 'permission', 'release', 'resouce' and 'role' will match the convention and thus have their cache entry in ehcache.xml
MinuteProject input
Input configuration
The analysis is resumed in the mp-config_LR.xml serving as input of MinuteProject
<!DOCTYPE root>
<generator-config>
    <configuration>
        <conventions>
            <target-convention type="enable-updatable-code-feature" />
        </conventions>       
        <model name="liferay" version="1.0" package-root="net.sf.mp.demo">
            <data-model>
                <driver name="mysql" version="5.1.16" groupId="mysql" artifactId="mysql-connector-java"></driver>
                <dataSource>
                    <driverClassName>org.gjt.mm.mysql.Driver</driverClassName>
                    <url>jdbc:mysql://127.0.0.1:3306/lportal</url>
                    <username>root</username>
                    <password>mysql</password>
                </dataSource>
                <primaryKeyPolicy oneGlobal="true">
                    <primaryKeyPolicyPattern name="none"></primaryKeyPolicyPattern>
                </primaryKeyPolicy>
            </data-model>
            <business-model>   
                    <generation-condition>
                    <condition type="exclude" startsWith="QUARTZ"></condition>
                    </generation-condition>
                <business-package default="liferay">
                    <condition type="package" startsWith="social" result="social"></condition>
                    <condition type="package" startsWith="user" result="user"></condition>
                    <condition type="package" startsWith="journal" result="journal"></condition>               
                </business-package>
                <enrichment>
                    <conventions>
                        <column-naming-convention type="apply-strip-column-name-suffix" pattern-to-strip="ID" />
                        <foreign-key-convention type="autodetect-foreign-key-based-on-similarity-and-map"
                            column-ending="id" column-starting="" column-body="match-entity">
                            <property tag="map-entity" name="user_" value="user"/>
                            <property tag="map-entity" name="group_" value="group"/>
                            <property tag="map-entity" name="organization_" value="organization"/>
                            <property tag="map-entity" name="permission_" value="permission"/>
                        </foreign-key-convention>
                        <reference-naming-convention type="apply-referenced-alias-when-no-ambiguity" is-to-plurialize="true" />       
                        <entity-content-type-convention type="apply-content-type-to-entity-ending-with" pattern="_" content-type="reference-data"></entity-content-type-convention>
                    </conventions>
                    <entity name="user_" alias="user"></entity>
                    <entity name="role_" alias="role"></entity>
                    <entity name="group_" alias="group"></entity>
                    <entity name="permission_" alias="permission"></entity>
                    <entity name="organization_" alias="organization"></entity>
                    <entity name="classname_" alias="classname"></entity>
                    <entity name="country" content-type="reference-data"></entity>
                </enrichment>
            </business-model>
        </model>
        <targets>
            <target refname="JPA2"
               fileName="mp-template-config-JPA2.xml"
               outputdir-root="../../dev/liferay/output/JPA2"
               templatedir-root="../../template/framework/jpa">
               <property name="add-querydsl" value="2.1.2"></property>
               <property name="add-jpa2-implementation" value="hibernate"></property>
               <property name="add-cache-implementation" value="ehcache"></property>
               
            </target>                        
            <target refname="LIB" fileName="mp-template-config-bsla-LIB-features.xml"
                templatedir-root="../../template/framework/bsla">
            </target>
    <target refname="CACHE-LIB" 
      fileName="mp-template-config-CACHE-LIB.xml" 
      templatedir-root="../../template/framework/cache">
    </target>            
        </targets>
    </configuration>
</generator-config>
Generation
Adapt the above config with the liferay database connection credentials.
You need version 0.7+ of minuteproject (and check that everything is fine by running the demos)
To generate place the following file mp-config_LR.xml into /mywork/config
Execute model-generation.(sh/cmd) mp-config_LR.xml

MinuteProject output
Generated artefacts
Entities
140+ entities have been created
12 Embeddable Id when composite primary have been found
Relationships
+- 300 OneToMany
+- 300 ManyToOne
+- 20 ManyToMany
Configuration
Entity Associate MetaModel for typesafe query
Maven pom.xml with querydsl integration
persistence.xml for test (local connection pool) and for release (jndi connection pool).
ehcache.xml

And All That in 10s less!
(local instance of mysql)


If you want to see what the resulting code looks like you can download on googlecode (version for minuteproject 0.8).
If you want to see what the Liferay datamodel looks like with relationships check here.


Test
But does it really work?
The sources are generated in /dev/liferay/output/JPA2
Writing a simple unit test in src/test/java which illustrates:
  • that you can write business added value
  • that querydsl for typesafe as an alternative to metamodel (MetaModel entities are also generated) is integrated
  • that ORM, ehcache configuration works
package my.liferay.test;

import java.util.List;

import javax.persistence.*;

import com.mysema.query.jpa.impl.JPAQuery;

import junit.framework.TestCase;
import net.sf.mp.demo.liferay.domain.liferay.Country;
import net.sf.mp.demo.liferay.domain.liferay.QCountry;

public class LiferayJPA2Test extends TestCase{
   
    public void testCountry() {
        // Initializes the Entity manager
        EntityManagerFactory emf = Persistence.createEntityManagerFactory("liferay");
        EntityManager em = emf.createEntityManager();
        EntityTransaction tx = em.getTransaction();

        //use Query dsl for type safe query
        JPAQuery query = new JPAQuery(em);
        QCountry qcountry = QCountry.country;
        List l = query.from(qcountry)
           .where(qcountry.countryid.lt(0))
           .list(qcountry);
        //remove the countries
        for (Country country:l) {
         em.remove(country);
        }
        // create a country
        Country c = new Country();
        // no primary policy key given so associate one manually 
        c.setCountryid(new Long(-100));
        c.setName("NEW-country");
        c.setA2("A2");
        c.setA3("A3");
        c.setActive(new Short("1"));
        c.setNumber("5");

        tx.begin();
        em.persist(c);
        tx.commit();

        em.close();
        emf.close();   
    }
}

Running with maven
Execute mvn package
This will compile and test the code.
In the build, querydsl classes will be generated and compiled
In the test, at run time, ORM and ehcache configuration will be loaded.

The sql statement issue by Hibernate at the end of the test is 
Hibernate:
    insert
    into
        country
        (a2, a3, active_, idd_, name, number_, countryId)
    values
        (?, ?, ?, ?, ?, ?, ?)
So yes, it works!
And package liferayBackEnd-1.0-SNAPSHOT.jar is created.

But what happens if there is a compile time or runtime issue with the generated code?
Do not worry the generated code is updatable which means that you can patch the code but your modification will be kept over successive generations. More information at http://minuteproject.wikispaces.com/Updatable_Generated_Code

Extend
Here the reverse-engineering target just one type of target JPA2 track.
Extend with other MinuteProject tracks
Example:
FitNessize you development: Adding the following snippet into the 'targets' node of the main configuration will generate an entire Fitnesse wiki ready to use for your acceptance testing.
   <target refname="FitNesse" 
      name="default" 
      fileName="mp-template-config-fitnesse.xml" 
      outputdir-root="../../dev/liferay/output/FITNESSE"
      templatedir-root="../../template/framework/fitnesse">
   </target>
For more information see this article
Extend you model
Enrich you model with:
  • views
  • store procedure
  • constraints
Minuteproject will generate relevant artefacts for them.
Conclusions
This article shows how to set reverse-engineering approach as a fast development methodology.
It is far less intrusive and restricted than Domain Driven Development and much faster.
This article also insists on how to apply your own conventions via 'declarative conventions'.

Minuteproject unleashes your productivity!
Things and tedious tasks that where bound to stay for the lifetime of your project can be removed.
At this point it takes more time for a DBA to write and execute an alter statement than to have full backend synchronisation!
Now you can move agile on your backend development.

Conclusion for the LifeRay Dev team:
Altering your backend might be quite long if you want to go to JPA2 track described above. Meanwhile you can use MinuteProject without any intrusivity on other tracks.
MinuteProject FitNesse fixture generated for Liferay and this really help increasing QA!
Try other MinuteProject tracks: for example any DB query can be RESTified (fully REST app generated) within seconds...

More information
More information about Minuteproject 4 JPA2 track can be found here.

Thursday, December 15, 2011

Adding Ehcache to Openxava application

Introduction
This article shows how to quickly enable Ehcache on Openxava applications and thus improving performance.
When viewing an entity and its graph, relationships are loaded. Adding a second-level cache fasten the retrieval of associated elements, since already loaded elements are retrieved from cache and not database.
Eventually this page explains how minuteproject treats this aspect with keeping its promise: write nothing.
As an example, we will take the Lazuly minuteproject showcase.
Openxava-Ehcache integration
In Openxava, you describe your model in the manner of Java annotated POJO.The annotations come from the standard JPA2 ORM and Openxava specific ones.
But nothing prevents you to add others. This is what is done to add caching. There are also couple of configurations to undertake to enable caching.
List of actions
  1. Add ehcache.xml config file at the root of your sources
  2. Modify persistence.xml to include second level cache
  3. Add caching annotation (alongside JPA2)
Remark:
Openxava comes with the ehcache.jar so there is no need to add a dependency.
Detailed actions
Add ehcache.xml
In /persistence place ehcache.xml file
<ehcache>
    <defaultCache
            maxElementsInMemory="1000"
            eternal="false"
            timeToIdleSeconds="300"
            timeToLiveSeconds="300"
            overflowToDisk="false"
            diskPersistent="false"
            diskExpiryThreadIntervalSeconds="300"
            memoryStoreEvictionPolicy="LRU"
            />
   <cache
    name="your.domain.object"
    maxElementsInMemory="5000"
    eternal="false"
    timeToIdleSeconds="300"
    timeToLiveSeconds="600"
    overflowToDisk="false"
   />
</ehcache>
Modify persistence.xml
Persistence.xml file contains information related to the persitence unit such as connection pool info, class or configuration to load. 'persistence.xml' is located in /persistence/META-INF
We will append properties for L2 cache.
        <properties>
            <property name="hibernate.dialect" value="org.hibernate.dialect.MySQLDialect"/>
            <property name="hibernate.cache.provider_class" value="net.sf.ehcache.hibernate.SingletonEhCacheProvider" />
            <property name="net.sf.ehcache.configurationResourceName" value="/ehcache.xml" />
            <property name="hibernate.cache.use_query_cache" value="true" />
            <property name="hibernate.cache.use_second_level_cache" value="true" />
            <property name="hibernate.generate_statistics" value="true" />   

        </properties>
Add cache annotation
Here the hibernate annotation is used instead of the standard one (Cacheable in fact seems not to work)
Place Cache annotation at class level of your domain object.
@org.hibernate.annotations.Cache(usage = org.hibernate.annotations.CacheConcurrencyStrategy.READ_WRITE)
Example
Lazuly application
Lazuly is a sample database holding conference information used for MinuteProject showcase purpose.
Minuteproject generates a comprehensive set of artefacts to speedup the release of OX application.
Further information can be found in Minuteproject 4 Openxava Lazuly showcase.
On this part we focus on the artefacts generated for the caching specific.
Minuteproject for the generation base itself on a configuration file, where we define the datamodel to reverse engineer. In this configuration there is an enrichement part where you can add information.
One of this information deals with the type of content is held in an entity. There are 4 possibilities (reference-data, master-data, pseudo-static-data, live-business-data)
If you enrich your entity with the content-type="master-data" or "reference-data" MinuteProject 4 Openxava will generate associated caching.
This is done here for the entity Country.
     <entity name="COUNTRY" content-type="reference-data">
Here are the cache related artefacts

ehcache.xml
<ehcache>

  <!--
    Sets the path to the directory where cache files are created.

    If the path is a Java System Property it is replaced by its value in the
    running VM.

    The following properties are translated:
    * user.home - User's home directory
    * user.dir - User's current working directory
    * java.io.tmpdir - Default temp file path

    Subdirectories can be specified below the property e.g. java.io.tmpdir/one
    -->
<!--MP-MANAGED-UPDATABLE-BEGINNING-DISABLE @ehcache-main-config-conference@-->
    <diskStore path="java.io.tmpdir"/>

 <!--
    Mandatory Default Cache configuration. These settings will be applied to caches
    created programmtically using CacheManager.add(String cacheName)
    -->
    <defaultCache
            maxElementsInMemory="1000"
            eternal="false"
            timeToIdleSeconds="300"
            timeToLiveSeconds="300"
            overflowToDisk="false"
            diskPersistent="false"
            diskExpiryThreadIntervalSeconds="300"
            memoryStoreEvictionPolicy="LRU"
            />
<!-- The unnamed query cache -->
   <cache
    name="org.hibernate.cache.StandardQueryCache"
    maxElementsInMemory="1000"
    eternal="false"
    timeToLiveSeconds="300"
    overflowToDisk="false"
   />
<!--MP-MANAGED-UPDATABLE-ENDING-->

<!--MP-MANAGED-UPDATABLE-BEGINNING-DISABLE @cache-entity-country-conference@-->
   <cache
    name="net.sf.mp.demo.conference.domain.admin.Country"
    maxElementsInMemory="5000"
    eternal="false"
    timeToIdleSeconds="300"
    timeToLiveSeconds="600"
    overflowToDisk="false"
   />
<!--MP-MANAGED-UPDATABLE-ENDING-->

<!--MP-MANAGED-ADDED-AREA-BEGINNING @custom-cache-definition@-->
<!--MP-MANAGED-ADDED-AREA-ENDING @custom-cache-definition@-->

</ehcache>
Persistence.xml
<persistence xmlns="http://java.sun.com/xml/ns/persistence"
             xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
             xsi:schemaLocation="http://java.sun.com/xml/ns/persistence http://java.sun.com/xml/ns/persistence/persistence_1_0.xsd"
             version="1.0">
             
    <!-- Tomcat + Hypersonic -->
    <persistence-unit name="default">
     <non-jta-data-source>java:comp/env/jdbc/conferenceDS</non-jta-data-source>
     <class>org.openxava.session.GalleryImage</class>
        <properties>
            <property name="hibernate.dialect" value="org.hibernate.dialect.MySQLDialect"/>
            <property name="hibernate.cache.provider_class" value="net.sf.ehcache.hibernate.SingletonEhCacheProvider" />
            <property name="net.sf.ehcache.configurationResourceName" value="/ehcache.xml" />
            <property name="hibernate.cache.use_query_cache" value="true" />
            <property name="hibernate.cache.use_second_level_cache" value="true" />
            <property name="hibernate.generate_statistics" value="true" />   
<!--MP-MANAGED-ADDED-AREA-BEGINNING @properties@-->
<!--MP-MANAGED-ADDED-AREA-ENDING @properties@-->
        </properties>
<!--MP-MANAGED-ADDED-AREA-BEGINNING @persistence-unit@-->
<!--MP-MANAGED-ADDED-AREA-ENDING @persistence-unit@-->
    </persistence-unit>       

<!--MP-MANAGED-ADDED-AREA-BEGINNING @persistence@-->
<!--MP-MANAGED-ADDED-AREA-ENDING @persistence@-->

</persistence>
Class annotation
@org.hibernate.annotations.Cache(usage = org.hibernate.annotations.CacheConcurrencyStrategy.READ_WRITE)
//MP-MANAGED-ADDED-AREA-BEGINNING @class-annotation@
//MP-MANAGED-ADDED-AREA-ENDING @class-annotation@
public class Country {

    @Hidden @Id @Column(name="id" )
    @GeneratedValue(strategy = GenerationType.AUTO)
    private Integer id; 
...
Generated code remark
The generated code has markers inside file extension comment.
Within MP-MANAGED-ADDED-AREA-BEGINNING and  MP-MANAGED-ADDED-AREA-ENDING you can place customized code
Within MP-MANAGED-UPDATABLE-BEGINNING-DISABLE and  MP-MANAGED-UPDATABLE-ENDING you can alter the code. To keep your modifications please change MP-MANAGED-UPDATABLE-BEGINNING-DISABLE into MP-MANAGED-UPDATABLE-BEGINNING-ENABLE.
Updatable code prevent you to lose your customisation over consecutive generations.
For more information on updatable code see Minuteproject updatable code.
Generation
  • Place the following file mp-config-LAZULY-OPENXAVA.xml in /mywork/config
  • on a prompt execute mp-model-generation(.sh/cmd) mp-config-LAZULY-OPENXAVA.xml 
  • the resulting artefacts in /DEV/output/openxava/conference 
To generate use the updated version of mp-config-LAZULY-OPENXAVA.xml
<!DOCTYPE root>
<generator-config>
 <configuration>
  <conventions>
   <target-convention type="enable-updatable-code-feature" />
  </conventions>  
  <model name="conference" version="1.0" package-root="net.sf.mp.demo">
   <data-model>
    <driver name="mysql" version="5.1.16" groupId="mysql" artifactId="mysql-connector-java"></driver>
    <dataSource>
     <driverClassName>org.gjt.mm.mysql.Driver</driverClassName>
     <url>jdbc:mysql://127.0.0.1:3306/conference</url>
     <username>root</username>
     <password>mysql</password>
    </dataSource>
    <!--
     for Oracle and DB2 please set the schema <schema> </schema>
    -->
    <primaryKeyPolicy oneGlobal="true">
     <primaryKeyPolicyPattern name="autoincrementPattern"></primaryKeyPolicyPattern>
    </primaryKeyPolicy>
   </data-model>
   <business-model>
    <!--
     <generation-condition> <condition type="exclude"
     startsWith="DUAL"></condition> </generation-condition>
    -->
    <business-package default="conference">
        <condition type="package" startsWith="STAT" result="statistics"></condition>
        <condition type="package" startsWith="COUNTRY" result="admin"></condition>
        <condition type="package" startsWith="ROLE" result="admin"></condition>    
    </business-package>
    <enrichment>
     <conventions>
      <column-naming-convention type="apply-strip-column-name-suffix"
       pattern-to-strip="_ID" />
      <reference-naming-convention
       type="apply-referenced-alias-when-no-ambiguity" is-to-plurialize="true" />
     </conventions>

     <entity name="COUNTRY" content-type="reference-data">
      <semantic-reference>
       <sql-path path="NAME" />
      </semantic-reference>
     </entity>
     <entity name="CONFERENCE_MEMBER">
      <semantic-reference>
       <sql-path path="FIRST_NAME" />
       <sql-path path="LAST_NAME" />
      </semantic-reference>
      <field name="STATUS">
       <property tag="checkconstraint" alias="conference_member_status">
        <property name="PENDING" value="PENDING" />
        <property name="ACTIVE" value="ACTIVE" />
       </property>
      </field>
      <field name="EMAIL">
       <stereotype stereotype="EMAIL" />
      </field>
     </entity>
     <entity name="SPEAKER">
      <field name="BIO">
       <stereotype stereotype="HTML_TEXT" />
      </field>
      <field name="PHOTO">
       <stereotype stereotype="PHOTO" />
      </field>
      <field name="WEB_SITE_URL">
       <stereotype stereotype="WEBURL" />
      </field>
     </entity>
     <entity name="PRESENTATION">
      <field name="STATUS">
       <property tag="checkconstraint" alias="presentation_status">
        <property name="PROPOSAL" value="PROPOSAL" />
        <property name="ACTIVE" value="ACTIVE" />
       </property>
      </field>
     </entity>
     <entity name="SPONSOR">
      <field name="STATUS">
       <property tag="checkconstraint" alias="sponsor_status">
        <property name="PENDING" value="PENDING" />
        <property name="ACTIVE" value="ACTIVE" />
       </property>
      </field>
      <field name="PRIVILEGE_TYPE">
       <property tag="checkconstraint" alias="sponsor_privilege">
        <property name="GOLDEN" value="Golden" />
        <property name="SILVER" value="Silver" />
        <property name="BRONZE" value="Bronze" />
       </property>
      </field>
     </entity>
     <!-- views -->
     <entity name="stat_mb_per_ctry_conf" alias="MEMBER_PER_COUNTRY_AND_CONFERENCE">
      <virtual-primary-key isRealPrimaryKey="true">
       <property name="virtualPrimaryKey" value="ID" />
      </virtual-primary-key>
     </entity>
     <entity name="stat_mb_by_role" alias="MEMBER_PER_ROLE_COUNTRY_AND_CONFERENCE">
      <virtual-primary-key isRealPrimaryKey="true">
       <property name="virtualPrimaryKey" value="id" />
      </virtual-primary-key>
      <field name="stat_mb_per_ctry_conf_ID" linkToTargetEntity="stat_mb_per_ctry_conf"
       linkToTargetField="id"></field>
     </entity>
    </enrichment>
   </business-model>
  </model>
  <targets>
   <!-- openxava -->
   <target refname="OpenXava" name="OpenXava"
    fileName="mp-template-config-openxava-last-features.xml"
    outputdir-root="../../DEV/output/openxava/conference"
    templatedir-root="../../template/framework/openxava">
   </target>

   <target refname="JPA2-LIB" fileName="mp-template-config-JPA2-LIB.xml"
    templatedir-root="../../template/framework/jpa">
   </target>
   
   <target refname="BSLA-LIB" fileName="mp-template-config-bsla-LIB-features.xml"
    templatedir-root="../../template/framework/bsla">
   </target>

   <target refname="CACHE-LIB" 
      fileName="mp-template-config-CACHE-LIB.xml" 
      templatedir-root="../../template/framework/cache">
   </target>
               
  </targets>
 </configuration>
</generator-config>

Test
To ensure that the caching is working properly:
  • Enable hibernate logging. Add the following snippet as extra properties in persistence.xml.
            <property name="hibernate.show_sql" value="true" />
            <property name="hibernate.format_sql" value="true" />
  • navigate to an entity that reference country (example Address)
  • When you view the detail of this entity you will notice that there is a load of the associated entity 'country'
  • But the second time you access to the details of this entity (or another entity referencing the same country instance), the country is not loaded twice from the database.

Tuesday, November 22, 2011

MinuteProject Telosys impedance matching

This article is to review couple of technical points of both generators, and see the different possibilities of integration.
My perception of Telosys is for the moment limited to what I learned at Devoxx presentation and what I've tried, so if I miss things, be keen to notify.

Telosys
  • Code generator
  • Based on DB
  • Performs enrichment via GUI(Eclispe plugin) in a phase called repository
  • Generates for a target framework (JEE, Ajax) developped by Telosys Team
  • Generates artefacts for entities (table) 
  • Metadata of template (name, ...) are store in text file
  • Use Velocity as a template rendering engine
Telosys goodies
  • JPA2 artefacts handle one2one relationship (not yet covered by minuteproject)
  • Telosys Plugin makes enrichment easy
Telosys neural
  • Targeting a specific 'Telosys Framework' solution as pros and cons
    • Pros 
      • fully customized
      • all in one inside IDE
    • Cons 
      • dev. burden on a team
      • no market place recognition
      • reusability of templates might be poor
Minuteproject
  • Code generator
  • Based on DB and WSDL based (next release: 0.7)
  • Performs enrichment via Configuration file or via a console
  • Generates for a range of market-place technologies
    • Spring/hibernate/ibatis/JPA1/JPA2
    • Openxava
    • Play
    • Grails
    • Roo
    • REST
    • Primefaces
    • FitNesse
    • ...
  • Generate for artifacts at 6 levels
    • Entity (tables/views)
    • Field
    • Package
    • Model
    • StoredProc
    • Application
  • Generated code can be updated and modifications are kept over consecutive generation
  • Metadata of template (name, 20 other attributes) are store in xml file 
  • Possibility to define convention:
    • Ex: view do not have PK but it is possible to define some individually or globally by applying a convention called 'apply-default-primary-key-otherwise-first-one'.
  • Use Velocity as a template rendering engine
Integration investigation tracks
Here are couple of feasible tracks under investigation:
  • Integrate Minuteproject tracks into Telosys
  • Add Telosys target as a MinuteProject 4 Telosys track
  • Add Telosys Plugin over Minuteproject generator

Monday, October 31, 2011

FitNessize your JEE dev with minuteproject


This article shows you how-to setup FitNesse in your development environment when dealing with Relation Database CRUD operations. MinuteProject 4 FitNesse generates entire set of FitNesse wiki pages and associated java fixtures allowing you to reset the database as well as performing intuitively CRUD operations on top of tables and Select on top of views.
To illustrate this MinuteProject track, the Database used comes from the Lazuly showcase.
The sources can be found under lazuly-fitnesse.
This page will show you:
  • FitNesse + Minuteproject operating mechanism in Agile development
  • How-to Generate CRUD fixture
  • Integrate then in your own scenario to gain QA
Overview




Install FitNesse and Generate A Custom CRUD wiki for you model
Prerequisites
  • Use Java 6
  • Install Lazuly DB (sql script here)
Installation
Download FitNesse
Execute java -jar fitnesse.jar
On a browser go to http://localhost to view the FitNesse wiki
Generate a FitNesse wiki for your model
  • Download MinuteProject
  • Generate using the following minuteproject configuration script: mp-config-LAZULY-FITNESSE.xml
  • Set this script in /mywork/config and run generate-model.(cmd/sh) mp-config-LAZULY-FITNESSE.xml
The mp-config-LAZULY-FITNESSE.xml configuration file
<!DOCTYPE root>
<generator-config>
 <configuration>
  <conventions>
   <target-convention type="enable-updatable-code-feature" />
  </conventions> 
  <model name="conference" version="1.0" package-root="net.sf.mp.demo">
   <data-model>
    <dataSource>
     <driverClassName>org.gjt.mm.mysql.Driver</driverClassName>
     <url>jdbc:mysql://127.0.0.1:3306/conference</url>
     <username>root</username>
     <password>mysql</password>
    </dataSource>
   </data-model>
   <business-model>
         <business-package default="conference">
             <condition type="package" startsWith="STAT" result="statistics"></condition>
             <condition type="package" startsWith="COUNTRY" result="admin"></condition>
             <condition type="package" startsWith="ROLE" result="admin"></condition>        
         </business-package>   
    <enrichment>
     <conventions>
         <view-primary-key-convention 
            type="apply-default-primary-key-otherwise-first-one" 
            default-primary-key-names="ID" >
         </view-primary-key-convention>
     </conventions>
    </enrichment>
   </business-model>
  </model>
  <targets> 
   <target refname="FitNesse" 
      name="default" 
      fileName="mp-template-config-fitnesse.xml" 
      outputdir-root="D:/DEV/LAZULY/lazuly-fitnesse"
      templatedir-root="../../template/framework/fitnesse">
   </target> 
   <target refname="LIB" 
      fileName="mp-template-config-bsla-LIB-features.xml" 
      templatedir-root="../../template/framework/bsla">
   </target>
  </targets>  

 </configuration>
</generator-config>
What is generated
Wiki Fixtures for CRUD operations


Wiki source to be copied in your FitNesse Root Directory
















FitNesse CRUD Wiki generated for you model.
Pick-up the snippet you need to:
  • SetUp you DB model
  • Perform sanity check 
    • after X UCs in a scenario, are my data correctly stored?
Lazuly FitNesse subwiki for select operations













Details of Select wiki fixtures for view 'stat_mb_by_role'
 
















Lazuly FitNesse subwiki for insert update delete operations











Details of Insert, Update, Delete wiki fixtures for table 'address'















Associated Java Fixture 























Assemble for your need
Build Java Fixture Project
Compile the code in an IDE
  • add src-generated to your source directories
  • add MP 4 FitNesse depencies libraries
    • For the moment they are provided in MP package under /target/fitnesse/ and /target/fitnesse/dep
  • Lazuly-fitnesse is shipped with an eclipse project that compiles the code to /bin
  • The compilation directory is to be reference by the FitNesse wiki path directives.
SetUp FitNesse Lazuly Wiki
Althought MinuteProject provides CRUD wiki ready to use, the purpose of this section is to setup a custom wiki that will be used for your UC.
  • Create your FitNesse Application Wiki HomePage
    • Edit main page add a section called 'FitNesseLazulyScenario'
      • Add snippet [[Lazuly][FitNesse.LazulyScenario]]
  • Click on Lazuly ? or go to http://localhost/FitNesse.LazulyScenario
  • Compose a simple structure
    • Simple structure proposal
      • HomePage section (define the path to use)
      • Setup section (Used to reset DB)
      • Data Definition section (Used to define common variables referenced in scenario)
      • Populate section (Used to initialize DB with data)
      • Couple of UC section
Write a sample scenario
This scenario will populate couple of tables of the Lazuly DB and check if the result expected in the views is correct.
HomePage section
Defines 
  • different pages (SetUp, VariableDefinition, UC sections) 
  • classpath

 
Setup section 
  • Reset Database
    • Reference Reset script executed by Scriptella (etl.xml)
      • Adapt the variable resetfilepathvar to point to your file path.
    • etl.xml references 3 sql script
      • reset_db.sql that proposed a Delete statement per table
        • It can be altered since the delete order might not be correct
        • In this case please add -- MP-MANAGED-STOP-GENERATING to prevent your modification to be lost for consecutive generation
      • insert_reference_data.sql
        • Add the data you wish
      • commit.sql that just perform a commit
  • Connection parameter to point to the database that include
    • DB URL
    • JDBC driver
    • Username
    • Password
  • All this code is generate in /ConferenceFitnesseWikiSetup/content.txt. It can also be copied from http://localhost/ConferenceFitnesseWikiSetup
 
Variable Definition section
Use Fitnesse directive !define to add relevant variable name for your scenario
Those variables will be later refered in your UCs

Populate section 
The SetUp page is automatically included at the beginning of your scenario.
But you have to reference the Variable Definition section. For that use the FitNesse directive !include (!include VariableDefinition)

Pick-up CRUD snippet from MP generated FitNesse Wiki to create you own customed DB initialization.
Insert reference data Country, Role
Insert business data: Address, Conference, ConferenceMember






 

















UC Check section
Pick-up CRUD snippet from MP generated FitNesse Wiki to create you own customed DB initialization.
In this case, I simply check that the data are stored correctly.
There is a view that gather BI information and I use it to validate the correctness of the information.


















Conclusion
This example shows that you can have use very easily CRUD fitnesse fixture (Zero LOC of development are required) customized for your data model.
To sum-up, here are the main points:
  • Reset DB
  • Initialize DB
  • Perform sanity check
  • Set-up FitNesse for your hadoc UCs for JEE data centric application


MinuteProject Spin-offs
At this point on you can guarante that the data retrieved from the view are correct.
That means that you have QA on your model.
If the views have a field containing unique values then you can benefit of MinuteProject spin-off i.e. use other track to generate out-of-the-box:
  • Web2.0 application in Openxava on top of views. See on this blog the following article 
  • REST application.
  • Web service application.
For Real UCs
Here the UCs demonstrated concern only CRUD operations on top of DB entities.
2 next articles in prepartion will show how to integrate with:
  • JPA2 ORM backend
  • Spring/JPA/Hibernate backend
Additional information
MinuteProject 4 FitNesse

Friday, October 14, 2011

MinuteProject goes SOA

Minuteproject "traditional" activity is to generate artifacts based on a Relation Database (RDBMS).
Meanwhile RDBMS is not the only type of standard structured information that is available on the market.
WSDL and WADL format describes a standard way to integrate systems with open technologies.
WSDL is widely used for webservice with SOAP messages integration promoted by the OASIS standards.
WADL is the representation of the REST contract offered by an application.

Minuteproject is integrating the metro API to parse wsdl and get a WsdlModel to generate artifacts upon.

The integration with metro is fairly simple

package metro;

import java.io.File;

import com.sun.tools.ws.processor.model.Model;
import com.sun.tools.ws.processor.modeler.wsdl.WSDLModeler;
import com.sun.tools.ws.wscompile.ErrorReceiverFilter;
import com.sun.tools.ws.wscompile.WsimportOptions;

public class SimpleWSDLModel {

public static void main(String[] args) {
ErrorReceiverFilter receiver = new ErrorReceiverFilter();
WsimportOptions options = new WsimportOptions();
options.addWSDL(new File(args[0]));
WSDLModeler wsdlModeler = new WSDLModeler(options, receiver);
Model wsdlModel = wsdlModeler.buildModel();
System.out.println(wsdlModel);
}
}

When giving a wsdl location file-system or URL, then Metro provides a wsdlModel that contains all the wsdl structure in memory and xml schema elements converted into their jaxb correspondance.
This eases the integration into minuteproject since the elements (jaxb classes, service, port, binding) are ready to be set into templates.
The goal for minuteproject is not to re-do the JAX-WS part of Metro available by wsimport command but to add a set of new tracks and artifacts to reduce the development effort.

Some tracks could be:
  • SOA FitNesse track: providing a FitNesse wiki with fixture (slim) to test the webservice implementation (without deployment)
  • Screens: JSF, vaadin for webservice client
  • WebService implementation
  • ...
This functionality will be available in the next release 0.7.
The SOA tracks will be added one at time.

Sunday, September 18, 2011

RESTication of Openxava app and Vaadin client

Article under construction

The purpose of this article is to provide REST interface for Openxava (OX) application.
It shows how-to
  • integrate jersey engine into the OX build
  • extends the JPA2 entities with JAXB and JAXRS annotations
  • write a little UC with methods that returns xml/json message
  • test it
  • Make a sample Vaadin client application for those REST services
To illustrate this, I will take an Openxava application generated by minuteproject (http://minuteproject.blogspot.com/2011/09/sql-select-to-web-application-in-couple.html)

Integrate Jersey engine in OX build

Jersey lib integration
Add the following libraries in your project /web/WEB-INF/lib
  • jsr305-1.3.2.jar
  • jersey-core-1.9-ea03.jar
  • jersey-server-1.9-ea03.jar
  • asm-3.1.jar (although asm.jar is shipped with OX this version is to used)
Here the dependency is on jersey-server-1.9-ea03

In your OX project add the file servlets.xml into /web/WEB-INF

   <servlet>
<servlet-name>petshop Jersey Web Application</servlet-name>
<servlet-class>com.sun.jersey.spi.container.servlet.ServletContainer</servlet-class>
<init-param>
<param-name>com.sun.jersey.config.property.resourceConfigClass</param-name>
<param-value>com.sun.jersey.api.core.PackagesResourceConfig</param-value>
</init-param>
<init-param>
<param-name>com.sun.jersey.config.property.packages</param-name>
<param-value>my.cool.report</param-value>
</init-param>
<load-on-startup>1</load-on-startup>
</servlet>

<servlet-mapping>
<servlet-name>petshop Jersey Web Application</servlet-name>
<url-pattern>/rest/*</url-pattern>
</servlet-mapping>

Openxava build takes what you define in servlets.xml file and insert it in the web.xml
The snippet above specifies the following:
  • Jersey engine will parse the package for JAXRS annotation in the package "my.cool.report"
  • the context path used for REST will be /rest/*
JAXB & JAXRS integration

Add Jaxb and Jaxrs annotation in OX jpa2 entities

Class MyConferenceCoolReport
package my.cool.report.conference.domain.report;

import java.sql.*;
import java.util.Date;
import java.util.List;
import java.util.ArrayList;
import java.util.Set;
import java.util.HashSet;

import javax.persistence.*;
import javax.ws.rs.*;
import javax.ws.rs.core.*;
import javax.xml.bind.annotation.*;

import my.cool.report.conference.dto.report.*;

import org.openxava.annotations.*;
import org.openxava.jpa.*;


/**
*
*

Title: MyConferenceCoolReport


*
*

Description: Domain Object describing a MyConferenceCoolReport entity


*
*/
@Entity (name="MyConferenceCoolReport")
@Table (name="my_conference_cool_report")
@Views({
@View(
name="base",
members=
""
+ "firstName ; "
+ "lastName ; "
+ "email ; "
+ "status ; "
+ "addressStreet1 ; "
+ "addressStreet2 ; "
+ "countryCode ; "
+ "conferenceName ; "
+ "conferenceBeginDate ; "
+ "conferenceEndDate ; "
),
@View(
name="Create",
extendsView="base"
),
@View(
name="Update",
extendsView="base",
members=
""
),
@View(extendsView="base",
members=
""
),
@View(name="myConferenceCoolReportDEFAULT_VIEW",
members=
" id ;"
+ "firstName ; "
+ "lastName ; "
+ "email ; "
+ "status ; "
+ "addressStreet1 ; "
+ "addressStreet2 ; "
+ "countryCode ; "
+ "conferenceName ; "
+ "conferenceBeginDate ; "
+ "conferenceEndDate ; "
)
})

@Tabs({
@Tab(
properties=
" firstName "
+", lastName "
+", email "
+", status "
+", addressStreet1 "
+", addressStreet2 "
+", countryCode "
+", conferenceName "
+", conferenceBeginDate "
+", conferenceEndDate "
)
,
@Tab(
name = "MyConferenceCoolReportTab",
properties=
" firstName "
+", lastName "
+", email "
+", status "
+", addressStreet1 "
+", addressStreet2 "
+", countryCode "
+", conferenceName "
+", conferenceBeginDate "
+", conferenceEndDate "
)
,
@Tab(
name = "MyConferenceCoolReportTabWithRef",
properties=
" firstName "
+", lastName "
+", email "
+", status "
+", addressStreet1 "
+", addressStreet2 "
+", countryCode "
+", conferenceName "
+", conferenceBeginDate "
+", conferenceEndDate "
)
})
@XmlAccessorType(XmlAccessType.FIELD)
@XmlType
@XmlRootElement
@Path ("/myreports")
public class MyConferenceCoolReport {

@Hidden @Id @Column(name="id" )
private Long id;

@Column(name="first_name", length=255, nullable=false, unique=false)
@Required
private String firstName;
@Column(name="last_name", length=255, nullable=false, unique=false)
@Required
private String lastName;
@Column(name="email", length=255, nullable=false, unique=false)
@Required
private String email;
@Column(name="status", length=45, nullable=false, unique=false)
@Required
private String status;
@Column(name="address_street1", length=255, nullable=true, unique=false)
private String addressStreet1;
@Column(name="address_street2", length=255, nullable=true, unique=false)
private String addressStreet2;
@Column(name="country_code", length=45, nullable=false, unique=false)
@Required
private String countryCode;
@Column(name="conference_name", length=255, nullable=false, unique=false)
@Required
private String conferenceName;
@Column(name="conference_begin_date", nullable=true, unique=false)
private Date conferenceBeginDate;
@Column(name="conference_end_date", nullable=true, unique=false)
private Date conferenceEndDate;

/**
* Default constructor
*/
public MyConferenceCoolReport() {
}

public Long getId() {
return id;
}

public void setId (Long id) {
this.id = id;
}

public String getFirstName() {
return firstName;
}

public void setFirstName (String firstName) {
this.firstName = firstName;
}
public String getLastName() {
return lastName;
}

public void setLastName (String lastName) {
this.lastName = lastName;
}
public String getEmail() {
return email;
}

public void setEmail (String email) {
this.email = email;
}
public String getStatus() {
return status;
}

public void setStatus (String status) {
this.status = status;
}
public String getAddressStreet1() {
return addressStreet1;
}

public void setAddressStreet1 (String addressStreet1) {
this.addressStreet1 = addressStreet1;
}
public String getAddressStreet2() {
return addressStreet2;
}

public void setAddressStreet2 (String addressStreet2) {
this.addressStreet2 = addressStreet2;
}
public String getCountryCode() {
return countryCode;
}

public void setCountryCode (String countryCode) {
this.countryCode = countryCode;
}
public String getConferenceName() {
return conferenceName;
}

public void setConferenceName (String conferenceName) {
this.conferenceName = conferenceName;
}
public Date getConferenceBeginDate() {
return conferenceBeginDate;
}

public void setConferenceBeginDate (Date conferenceBeginDate) {
this.conferenceBeginDate = conferenceBeginDate;
}
public Date getConferenceEndDate() {
return conferenceEndDate;
}

public void setConferenceEndDate (Date conferenceEndDate) {
this.conferenceEndDate = conferenceEndDate;
}

@Transient
@GET
@Path("{reportId}")
@Produces ({MediaType.APPLICATION_XML, MediaType.APPLICATION_JSON})
public MyConferenceCoolReport findById (@PathParam ("reportId") Long reportId) {
EntityManager em = XPersistence.getManager();
MyConferenceCoolReport p = em.find(MyConferenceCoolReport.class, reportId);
return p;
}

@Transient
@GET
@QueryParam("{countryCode}")
@Produces ({MediaType.APPLICATION_XML, MediaType.APPLICATION_JSON})
public MyConfReports find (@QueryParam ("countryCode") String countryCode) {
String q = null;
EntityManager em = XPersistence.getManager();
if (countryCode!=null) {
q = "select m from MyConferenceCoolReport m where countryCode = '"+countryCode+"'";
} else
q = "select m from MyConferenceCoolReport m ";
Query query = em.createQuery(q);
List<myconferencecoolreport> l = query.getResultList();
MyConfReports m = new MyConfReports();
m.setMyCoolReport(l);
return m;
}

}

Annotation of entities
The previous snippet indicates that entity MyConferenceCoolReport contains a transient method (findById) use for REST GET operation and using the param reportId to load an entity. It can be returned either as Xml or in a Json format.
It contains also a method find that takes a parameter a country code and return a DTO based on the class MyConfReports

package my.cool.report.conference.dto.report;
package my.cool.report.conference.dto.report;

import java.util.*;

import javax.xml.bind.annotation.*;

import my.cool.report.conference.domain.report.*;

@XmlAccessorType(XmlAccessType.FIELD)
@XmlType
@XmlRootElement
public class MyConfReports {

private List<MyConferenceCoolReport> myCoolReport;

public MyConfReports () {
}

public List<MyConferenceCoolReport> getMyCoolReport() {
return myCoolReport;
}

public void setMyCoolReport(List<MyConferenceCoolReport> myCoolReport) {
this.myCoolReport = myCoolReport;
}

}
Deploy
Use the ant build on your project.
Remark: do not forget when using openxava with eclipse to set "Build Automatically" on the "Project" tab.

Testing
Via the browser type
http://localhost:8080/conference/rest/myreports?countryCode=BE to see all the


http://localhost:8080/conference/rest/myreports/ it returns in an XML format.




Vaadin app client
//todo

Conclusion
//todo


Saturday, September 3, 2011

SQL select to web application in couple of seconds

Maybe you are not aware but if you just know how to write one sql select statement, you're just couple of seconds from having an web application and couple of minutes from enjoying it as a Liferay portlet.
The key ingredients for this recipe are: Openxava, Liferay and ... Minuteproject.

The equation is:
sql select statement (containing one field considered as unique) = Openxava (Webapp) + Liferay (Portlet)
in less than 5 minutes.

Prerequisites
  • Download Openxava (4.2.2), Liferay, Minuteproject (0.5.7.1+)
  • Set environment variables OX_HOME and MP_HOME to Openxava and Minuteproject home directories respectively.
  • Use Java 6
Create your statement
  • Create a sql select statement
  • Ensure that one of the return field can be considered as unique (if not add one)
  • Create a view from your statement
    • Ex: CREATE VIEW abc as
Remark:
Why is it necessary to have a unique field?
It is to ensure that MP can place a @Id JPA2 annotation on one field of the generated entity.
It has to be unique to avoid issue dealing with first level cache.

Start Minuteproject console
  • Point to your model and fill your model name (ex: efg)
  • select your view abc (in customisation tab)
  • add the unique field as 'virtual' primary key. Your unique field is not a PK but it will acts as to uniquely identify a record entry.
  • Choose target openxava and click generate.
Release your application
  • go to the generated code
  • check environment variables (OX_HOME, MP_HOME) are set.
  • execute from a command prompt: build-efg.cmd/sh
  • depending on the OS from couple of seconds to a dozen of seconds and a browser should open to the address http://localhost:8080/egf/xava/homeMenu.jsp
Deploy on Liferay
  • go to OX_HOME/workspace.dist/efg.dist/efg.war
  • add ejb.jar and jdbc driver jar in LIFERAY_HOME/tomcat/lib
  • start Liferay
  • Upload your portlet
  • Install your portlet
  • Enjoy
Remark:
Whether on the standalone tomcat app or on the Liferay deployed portlet, Openxava is only offering the user selection. It is normal since the DB object is a view, Minuteproject 4 Openxava suppress the Create/Update/Delete, just the Select functionality is kept.


Example

As model I take the lazuly model project in version 0.1 it contains 13 tables.
The sql to create the schema on mysql can be downloaded at http://code.google.com/p/lazuly/source/browse/trunk/conference-db/conference-db-mysql-0.1.sql

Create your view from your sql select statement


Start MinuteProject console


Scope your model


Generate
Choose your technology: here Openxava, and click on Generate.
The code will be generated in MP_HOME/output/conference/OpenXava

Make a standalone Openxava app
Go to the generated code
Open a prompt
Check or set OX_HOME and MP_HOME
execute build-conference.cmd/sh
Couple of seconds later a browser should open to the model (conference) menu.
Report tab has one entry 'my conference cool report'.
When clicking on it here is a sample screen with data that you could see:



Deploy on Liferay
  • Add the datasource (the snippet for tomcat is in OX_HOME/workspace/conference/OpenXava/openxava/other/tomcat/snippet/conference_jndi_for_tomcat_snippet.txt is to be inserted into LIFERAY_HOME/tomcat/conf/context.xml)
  • Upload the portlet war (OX_HOME/workspace.dist/conference.dist/conference.war
  • Position it
  • Enjoy.




Other information

You can also find other information regarding Minuteproject 4 Openxava generation at

Conclusion
  • Your model is your technology tutorial
  • Productivity can be greatly improved
  • Data Quality and troubleshooting operations are quickly available
  • Keep on improving send feed-back to minuteproject@gmail.com or tweet @minuteproject

Tuesday, August 23, 2011

conscious nerd?

After all, at the end of the day, what are we geek for?

Yes folks, we spend hours, days, years of contributing, criticizing, publishing but what is worth for?
What is our mission(s) for those sleepless nights or more pragmatically for this shadow IT? (Yes, we all know that we do ;) )
Is it pride, self sufficience, been the first, be part of the wave or the be the wave itself that drives us?
I do not pretend to introspect nor to find your reason for a free contribution and commitment. I just heap up my own modest contribution to the stack, and wonder where this amont of info will lead us to?
Personally, I think that IT should gently impact our way of living (hey not (only) in the way of booking cheaper holidays...) but in a way to share knowledge and get conscience.

Sharing knowledge
: we've been granted by lots of it and we emphasize with our own to grow it bigger and wider.
Promote Altruism: Give the community without expectation!
It may sound naive at first, but it's really a driving force!
You will be remembered! (BTW, The long tail pattern is here for you!)).

Get conscience: Although our projects, contributions might die or succeed, we are part of an effort that probably transcends us and would lead to a better living and understanding of the position and possibility/feasibilities offered to us.
By position, I mean as an actor to the web and community in the present.
By possibility/feasibility, I mean the opportunities that will make our destinity brights stronger. Yes IT will lead us to Mars! and the solar system...

Future: is what we decide! (mitigating the impondarables ;)


Saturday, August 6, 2011

Project development golden pentagon

Intro

This article is about the 5 golden principles that lead to success the development of a product.
It also gives pragmatic development tips to streamline the development process.

Golden Pentagon
The 5 major objectives that any software project should fulfill are to deliver a solution that
  • matches business requirements
  • is operationable
  • is within budget
  • is within scheduled timeframe
  • is maintable
Match Business Requirements
Been out of scope is obviously useless. So the closest the better. But which methodology to use???
The IT world is full of acronyms and methodology all in one solution but what I recommand is the following:
  • AVOID RUP!!!
It brings nothing except troubles and an ironical conclusion: if it fails it means that you did not follow the methodology correctly.
  • CHOOSE AGILE
But here again the goal is to produce something and not to be lost within scrums, sprints, backlogs and burndown curves.
So how to concretely describe the business requirements in order fulfill the product owner desires? And how to make those requirement be in line with the Truth (the stuff that will run in production)?
  • CHOOSE FITNESSE
Write stories to describe your business requirements make and exhaustive list of them. Reach the developement contract accord between project owner, developers, analysts, tester, project manager. Since is wiki human readable it is feasible.
Write the correct fixtures to loosely couple your business requirements with the underlying code.
This will bodyguard your development effort with Quality Assurance.
Meanwhile to put that in place there is some learning curve and some fixture development time effort. To easy this part MP provides some ready to use fixtures.

Minuteproject asset:
When working with a Datamodel, MP generates an entire wiki with associated fixtures to make crud operation easy.
Example :
  • you need to illustrate a scenario, in the set up use MP fixtures to reset and populate the database.
  • after couple of features performing a suite of UCs you need to have a sanity check in the database (to check the data is correctly stored). MP generates the Fitnesse fixture for the desired sanity check query materialized in a DB object view.
A specific entry for the MP 'Lazuly' showcase will be put in this blog soon. Stay tuned.

Be operationable
What's the point to develop something that runs properly. This is a wide concept be in the terms of web development is to have a solution that scales properly, does not be to be restarted (no memory leaks), sustain concurrency and responds fast.

Stay within budget
Also obvious and speaking from itself, it is a reality that most project forget and tend to counterbalance by betting on CR (Change request), i.e. anappropiate analysis leading to clumsy UCs raise profitable CRs.
When the analysis tackles with model (Change/enhancement), MP is ready to cope with it with extrem performance (just a couple of seconds) to have your backend aline/online.


Stay within deadlines
What is the interest of releasing after deadlines (or what is the interest of setting dealines if we know from the start there're just virtual?????).
By remove a huge dev burden, MP helps you handle deadline easily. From to 40% to 60% of your developement effort can be handle by MP according the tracks you use.
Write less code.
WRITE ONLY THE CODE YOU NEED.
Do not overcode.



Think future: stay maintable



//this slot is under construction, wait a while for it to be finished... there are some many things to say!!!

Tuesday, July 19, 2011

3 features I would dream to see in Openxava

Pull BI out of the ghetto
Currently 'all in one' framework such as grails, playframework, openxava, roo, rails behave all more or less the same for CRUD modal navigation
Start from a list (resulting from a search screen or default filtering), then goes on one entity to manipulate.
Personally, I think there is/should be a step before. I would like to see the big picture instead of scrolling down a list or ajusting search params.
Furthermore, this overview can be express in format of a graphical chart which has a greater expressing power than looking at records.

To give a concrete example:
I design a 'helpdesk app' and one entity is called 'issue'. Issue entity has a number of fields among those 'type', 'status', 'origin', 'date-creation', 'date-resolution'.
Would'nt it be nice to visualize:
  • on a piechart the ventilation by type/status or/and origin, and then click on one portion to get the result filtered accordingly;
  • on a trend chart the evolution of the issue by type/status or/and origin against 'date-creation', 'date-resolution' before drilling down into the entries.
Those information come from BI queries that should be added to ease navigation and clarify the business. I found more attractive to come from a dashboard representing an overview of the business than being stick to the detail in an early phase.
It would bridge the gap between the BI world an the 'attached-entity' centric paradigm.

For Openxava, we could easily imagine other annotation

@dashboards
@dashboard (representation="piechart", filtering="type,status,origin")
@dashboard (representation="trend", filtering="type,status,origin", start-period="date", period-sample='day/month/year', time-param="date-creation)

Content search

Framework search as compass or hibernate-search propose some content search solution based on annotations.
If we use compass we can imagine that entity tagged as @searchableRoot could have a free text search for the field marked as @searchable.

GWT - ideally Vaadin rendering

It speaks from itself.
Get the benefit of other community for front-end snippets.
Integration with GAE, google maps...

And minuteproject in all that
Well, all that is marked as annotation could be described as model enrichment when it is data centric.
So we could easily imagine to have a field enrichment stating that it belongs to a dashboard, or better have a global convention that applies to the model that states that all fields ending with type or status are part of a dashboard. In this case one line of configuration is needed to impact the entire navigation model.
The same is true for content search but here the enrichment is already available for minuteproject4Solr track.

Monday, July 11, 2011

JPA2 simple reverse-engineering via MP console

This article shows how quickly you can reverse-engineering for JPA2 via minuteproject console.

The console is just find to have a quick look at what you can expect.

The console has two catalogs:
  • One for the target DB (for the moment there are 4 of them Oracle, DB2, mysql, hsqldb)
  • One for the target technologies (there are couples such as JPA2, spring/hibernate/Jpa/DAO, grails, playframework, vaadin).
The DB catalog holds for each db parameters corresponding to its default configurations such as:
  • primary key (Oracle use sequence, Mysql autoincrement)
  • dialect (for hibernate or orm vendor specific)
  • jdbc driver class
  • maven driver artifacts
To add new DB just add some nodes in /catalog/database-catalog.xml and restart the console.
You can share your new configuration for DB with minuteproject to be included in the next release.

The technology catalog holds for each track the set of artifacts for the different level (field, entity, package, model, application) that is to be generated.

Here a little video illustrating how to use the console:

MinuteProject 4 JPA2: reverse-engineering via console from minuteproject on Vimeo.


If you want additional feature you have to use Minuteproject with a configuration file.


Monday, June 27, 2011

MinuteProject 4 JPA2: Lazuly showcase

Intro
The goal of this article is to have a working JPA2 layer on top of a data model by writting 0 (yes ZERO) lines of code.
Lazuly-db is a opensource database hosted on googlecode that represents the structure of a database holding conference related informations.
Lazuly-db will be used as a showcase of various MinuteProject technology tracks.
In version 0.1 it has 13 tables, 2 views.

This article explains the different steps to quickly have a working JPA2 backend.
It is furthermore a tutorial on how to enrich the database model to get customized artifacts providing specific behaviour.
The purpose is to give to user enough material to customize it and go on extending your model.

MinuteProject short story
The idea behind minuteproject is to
  • go fast;
  • reduce the technology learning curve by knowledge cristalization;
  • improve the quality of the artifact;
  • shorten drastically the time to market of the product.
Minuteproject acts as a Productivity provider targeting multiple aspects of the reverse-engineering for a target architecture (here JPA2).
While most of existing reverse-engineering solution focus on one part of your architecture (Domain Object), MinuteProject can generate artifacts related to:
  • columns;
  • entities (tables,views);
  • stored procedures;
  • package;
  • model;
  • application.
It allows the developer to achieve 'reproductability' that is a great complement of reusability.
Reusability allows to bundle common behavior into a package/framework/specification in order to use them but not to code them.
Reproductability allows to reach the same quality of artifacts (config/classes...) matching a target architecture (reused) although the context changes (DB model).

JPA2 track
JPA2 is a JEE specification for persisting objects in a database.
MinuteProject 4 JPA2 (release 0.5.5) offers a reverse-engineering solution targeting JPA2 artifacts:
  • Java Entity classes for tables and views;
  • JPA2 metamodel;
  • persistence xml;
  • maven pom.xml;
  • vendor specific implementation;
  • querydsl integration.
More information can be found at minuteproject4jpa2

Operating principle
MinuteProject works with a configuration file.
This file holds the information regarding the model and how to enrich it. It also specifies the target bundle of templates for a technology track.

Datasource
Lazuly DB main entities: The schema below resumes the lazuly DB main entities (tables/views) and theirs relationships.


This part summarizes the different configurations used:

Here is the sample xml configuration that is applied to the model
<!DOCTYPE root>
<generator-config>
<configuration>
<model name="conference" version="1.0" package-root="net.sf.mp.demo">
<!-- <model name="lazuly" version="1.0" package-root="mp.demo"> -->
<data-model>
<driver name="mysql" version="5.1.16" groupId="mysql"
artifactId="mysql-connector-java"></driver>
<dataSource>
<driverClassName>org.gjt.mm.mysql.Driver</driverClassName>
<url>jdbc:mysql://127.0.0.1:3306/conference</url>
<username>root</username>
<password>mysql</password>
</dataSource>
<!-- for Oracle and DB2 please set the schema <schema> </schema> -->
<primaryKeyPolicy oneGlobal="true">
<primaryKeyPolicyPattern name="autoincrementPattern"></primaryKeyPolicyPattern>
</primaryKeyPolicy>
</data-model>
<business-model>
<!-- <generation-condition> <condition type="exclude" startsWith="DUAL"></condition>
</generation-condition> -->
<business-package default="conference">
<condition type="package" startsWith="STAT" result="statistics"></condition>
<condition type="package" startsWith="COUNTRY" result="admin"></condition>
<condition type="package" startsWith="ROLE" result="admin"></condition>
</business-package>
<enrichment>
<conventions>
<column-naming-convention type="apply-strip-column-name-suffix"
pattern-to-strip="_ID" />
<reference-naming-convention
type="apply-referenced-alias-when-no-ambiguity" is-to-plurialize="true" />
</conventions>

<entity name="PRESENTATION">
<field name="STATUS">
<property tag="checkconstraint" alias="presentation_status">
<property name="PROPOSAL" value="PROPOSAL" />
<property name="ACTIVE" value="ACTIVE" />
</property>
</field>
</entity>
<entity name="SPONSOR">
<field name="STATUS">
<property tag="checkconstraint" alias="sponsor_status">
<property name="PENDING" value="PENDING" />
<property name="ACTIVE" value="ACTIVE" />
</property>
</field>
<field name="PRIVILEGE_TYPE">
<property tag="checkconstraint" alias="sponsor_privilege">
<property name="GOLDEN" value="Golden" />
<property name="SILVER" value="Silver" />
<property name="BRONZE" value="Bronze" />
</property>
</field>
</entity>
<!-- views -->
<entity name="stat_mb_per_ctry_conf" alias="MEMBER_PER_COUNTRY_AND_CONFERENCE">
<virtual-primary-key isRealPrimaryKey="true">
<property name="virtualPrimaryKey" value="ID" />
</virtual-primary-key>
</entity>
<entity name="stat_mb_by_role" alias="MEMBER_PER_ROLE_COUNTRY_AND_CONFERENCE">
<virtual-primary-key isRealPrimaryKey="true">
<property name="virtualPrimaryKey" value="id" />
</virtual-primary-key>
<field name="stat_mb_per_ctry_conf_ID" linkToTargetEntity="stat_mb_per_ctry_conf"
linkToTargetField="id"></field>
</entity>
</enrichment>
</business-model>
</model>
<targets>

<target refname="JPA2" fileName="mp-template-config-JPA2.xml"
outputdir-root="../../dev/output/JPA2" templatedir-root="../../template/framework/jpa">
<property name="add-querydsl" value="2.1.2"></property>
<property name="add-jpa2-implementation" value="hibernate"></property>
</target>

<target refname="LIB" fileName="mp-template-config-bsla-LIB-features.xml"
templatedir-root="../../template/framework/bsla">
</target>

</targets>
</configuration>
</generator-config>
The configuration provides:
  • A model name: conference, version and package root;
  • the driver as maven reference;
  • connection parameters;
  • primarykey policy (autoincrement)
  • business packages: entities starting with stat go to 'statistics'...;role, countries goes in administration package
  • convention: strip-column-name-suffix _ID (by convention in the database, I have column naming ending with '_ID' for foreign key. Here the Java naming convention can defer from the DB naming
  • convention: apply-referenced-alias-when-no-ambiguity indicating that when a table has a one2many relationship with other (i.e. has child), if it has not 2 children with the same other side table (no ambiguity) than the name of the variable is based on the other side table, and plurialized (english) collections
  • Entity enrichment: name alias (can have a distinct Java class name than the one derived from the DB);
  • Field enrichment: provide a check constraint that becomes a Java Enum
  • Entity (views) enrichment by providing them a primary key (create a unique key) by adding foreignkey.
The target part of the configuration specifies against which target technology, MinuteProject has to generate. Here the track is JPA2, where it is specified to use querydsl 2.1.2 and a JPA vendor implementation Hibernate.
Those indications are used to generate pom.xml and persistence.xml files.

Sources and installation
Lazuly project
Svn the source as describe on http://code.google.com/p/lazuly/source/checkout

Mysql
Install mysql
run conference-db.0.1.sql on schema conference

Minuteproject
Download and install minuteproject 0.5.5 or higher. Minuteproject is installed in MP_HOME.

Execution
Normally within couple of seconds (1 to 5) the code should be generated (If DB instance and minuteproject run is on same machine)

Alternate execution
  • Open minuteproject console %MP_HOME%/bin/start-console.(cmd/sh)
  • field the connection parameters
  • add package root and model name
  • click generate
This alternative generation works fine to give a first glance at what you can have as output. A better enrichment is available via the xml config as previously described.

Output
The output can be browsed at http://code.google.com/p/lazuly/source/browse/#svn%2Ftrunk%2Flazuly-jpa2

Here are some samples of what has been generated in terms of

Structure
Here the structure open with as an eclipse-maven project


Java class as JPA2 entity with enumeration, OneToMany, ManyToOne, ManyToMany
You can note that the field 'abstract' in the table 'presentation' is converted to 'abstractName' to avoid compilation error. This is true for all the java reserved keywords. The getter and setter are adapted accordingly.
package net.sf.mp.demo.conference.conference;

import java.sql.*;
import java.util.Date;
import java.util.List;
import java.util.ArrayList;
import java.util.Set;
import java.util.HashSet;

import javax.persistence.*;

import net.sf.mp.demo.conference.conference.Evaluation;
import net.sf.mp.demo.conference.conference.PresentationPlace;
import net.sf.mp.demo.conference.conference.Speaker;
import net.sf.mp.demo.conference.enumeration.conference.PresentationStatusEnum;

/**
*
* <p>Title: Presentation</p>
*
* <p>Description: Domain Object describing a Presentation entity</p>
*
*/
@Entity (name="Presentation")
@Table (name="presentation")
public class Presentation {

@Id @Column(name="id" )
@GeneratedValue(strategy = GenerationType.AUTO)
private Long id;

@Column(name="start_time", nullable=false, unique=false)
private Timestamp startTime;
@Column(name="end_time", nullable=false, unique=false)
private Timestamp endTime;
@Column(name="abstract", length=500, nullable=false, unique=false)
private String abstractName;
@Column(name="title", length=255, nullable=false, unique=false)
private String title;
@Enumerated (EnumType.STRING)
private PresentationStatusEnum status;
@Column(name="proposal_time", nullable=true, unique=false)
private Timestamp proposalTime;

@ManyToOne (fetch=FetchType.LAZY )
@JoinColumn(name="presentation_place_id", referencedColumnName = "id", nullable=true, unique=false )
private PresentationPlace presentationPlaceId;

@OneToMany (targetEntity=net.sf.mp.demo.conference.conference.Evaluation.class, fetch=FetchType.LAZY, mappedBy="presentationId", cascade=CascadeType.REMOVE)//, cascade=CascadeType.ALL)
private Set <Evaluation> evaluations = new HashSet<Evaluation>();

@ManyToMany
@JoinTable(name="SPEAKER_PRESENTATION",
joinColumns=@JoinColumn(name="presentation_id"),
inverseJoinColumns=@JoinColumn(name="speaker_id")
)
private Set <Speaker> speakers = new HashSet <Speaker> ();

/**
* Default constructor
*/
public Presentation() {
}

public Long getId() {
return id;
}

public void setId (Long id) {
this.id = id;
}

public Timestamp getStartTime() {
return startTime;
}

public void setStartTime (Timestamp startTime) {
this.startTime = startTime;
}

public Timestamp getEndTime() {
return endTime;
}

public void setEndTime (Timestamp endTime) {
this.endTime = endTime;
}

public String getAbstractName() {
return abstractName;
}

public void setAbstractName (String abstractName) {
this.abstractName = abstractName;
}

public String getTitle() {
return title;
}

public void setTitle (String title) {
this.title = title;
}

public PresentationStatusEnum getStatus() {
return status;
}

public void setStatus (PresentationStatusEnum status) {
this.status = status;
}

public Timestamp getProposalTime() {
return proposalTime;
}

public void setProposalTime (Timestamp proposalTime) {
this.proposalTime = proposalTime;
}

public PresentationPlace getPresentationPlaceId () {
return presentationPlaceId;
}

public void setPresentationPlaceId (PresentationPlace presentationPlaceId) {
this.presentationPlaceId = presentationPlaceId;
}

public Set<Evaluation> getEvaluations() {
if (evaluations == null){
evaluations = new HashSet<Evaluation>();
}
return evaluations;
}

public void setEvaluations (Set<Evaluation> evaluations) {
this.evaluations = evaluations;
}

public void addEvaluations (Evaluation evaluation) {
getEvaluations().add(evaluation);
}

public Set<Speaker> getSpeakers() {
if (speakers == null){
speakers = new HashSet<Speaker>();
}
return speakers;
}

public void setSpeakers (Set<Speaker> speakers) {
this.speakers = speakers;
}

public void addSpeakers (Speaker speakers) {
getSpeakers().add(speakers);
}
}

Enumeration
You can note the 'equals (string s)' method which is quite handy.
package net.sf.mp.demo.conference.enumeration.conference;

import java.util.ArrayList;
import java.util.List;

public enum SponsorPrivilegeTypeEnum {

GOLDEN("Golden"),
SILVER("Silver"),
BRONZE("Bronze");

private final String value;

SponsorPrivilegeTypeEnum(String v) {
value = v;
}

public String value() {
return value;
}


public static SponsorPrivilegeTypeEnum fromValue(String v) {
for (SponsorPrivilegeTypeEnum c : SponsorPrivilegeTypeEnum.values()) {
if (c.value.equals(v)) {
return c;
}
}
return null;
}


/**
* Return a list that contains all the enumeration values
* @return List<SponsorPrivilegeTypeEnum> the that contains all the enumeration values
*/
public static List<SponsorPrivilegeTypeEnum> getList() {
List<SponsorPrivilegeTypeEnum> list = new ArrayList<SponsorPrivilegeTypeEnum>();
for (SponsorPrivilegeTypeEnum c : SponsorPrivilegeTypeEnum.values()) {
list.add(c);
}
return list;
}

/**
* @param value
* @return boolean : true if the value exist in the enum
*/
public static boolean contains (String value) {
for (SponsorPrivilegeTypeEnum c : SponsorPrivilegeTypeEnum.values()) {
if (c.toString().equals(value))
return true;
}
return false;
}

public static boolean containsValue (String value) {
for (SponsorPrivilegeTypeEnum c : SponsorPrivilegeTypeEnum.values()) {
if (c.value().equals(value))
return true;
}
return false;
}

public boolean equals (String s) {
if (s==null) return false;
return s.equals(value);
}

}
Java class as JPA2 entity for view
You can note the enrichment performed is translated into different Java and DB naming conventions for the entity itself as well as its dependencies.
Now you can use the view for all the JPA2 select operations.
package net.sf.mp.demo.conference.statistics;

import java.sql.*;
import java.util.Date;
import java.util.List;
import java.util.ArrayList;
import java.util.Set;
import java.util.HashSet;

import javax.persistence.*;

import net.sf.mp.demo.conference.statistics.MemberPerRoleCountryAndConference;

/**
*
* <p>Title: MemberPerCountryAndConference</p>
*
* <p>Description: Domain Object describing a MemberPerCountryAndConference entity</p>
*
*/
@Entity (name="MemberPerCountryAndConference")
@Table (name="stat_mb_per_ctry_conf")
public class MemberPerCountryAndConference {

@Id @Column(name="id" ,length=300)
private String id;

@Column(name="country", length=45, nullable=false, unique=false)
private String country;
@Column(name="conference_name", length=255, nullable=false, unique=false)
private String conferenceName;
@Column(name="number", nullable=false, unique=false)
private Long number;

@OneToMany (targetEntity=net.sf.mp.demo.conference.statistics.MemberPerRoleCountryAndConference.class, fetch=FetchType.LAZY, mappedBy="statMbPerCtryConfId", cascade=CascadeType.REMOVE)//, cascade=CascadeType.ALL)
private Set <MemberPerRoleCountryAndConference> memberPerRoleCountryAndConferences = new HashSet<MemberPerRoleCountryAndConference>();

/**
* Default constructor
*/
public MemberPerCountryAndConference() {
}

public String getId() {
return id;
}

public void setId (String id) {
this.id = id;
}

public String getCountry() {
return country;
}

public void setCountry (String country) {
this.country = country;
}

public String getConferenceName() {
return conferenceName;
}

public void setConferenceName (String conferenceName) {
this.conferenceName = conferenceName;
}

public Long getNumber() {
return number;
}

public void setNumber (Long number) {
this.number = number;
}

public Set<MemberPerRoleCountryAndConference> getMemberPerRoleCountryAndConferences() {
if (memberPerRoleCountryAndConferences == null){
memberPerRoleCountryAndConferences = new HashSet<MemberPerRoleCountryAndConference>();
}
return memberPerRoleCountryAndConferences;
}

public void setMemberPerRoleCountryAndConferences (Set<MemberPerRoleCountryAndConference> memberPerRoleCountryAndConferences) {
this.memberPerRoleCountryAndConferences = memberPerRoleCountryAndConferences;
}

public void addMemberPerRoleCountryAndConferences (MemberPerRoleCountryAndConference memberPerRoleCountryAndConference) {
getMemberPerRoleCountryAndConferences().add(memberPerRoleCountryAndConference);
}

}

Run
Since it generate a pom.xml, download maven and run 'mvn install'
It generates the lazuly backend jar containing the entities, the metamodel and the querydsl classes.