ERP/2.50/Developers Guide/Concepts/Development Model

From InfiniteERP Wiki
Jump to: navigation, search

{{#hierarchy-top:}}

Introduction

The Openbravo Development Environment (abbreviated as ODE) is a framework of tools, methodologies and processes to make Openbravo ERP development process easier and to help developers to be more efficient in tasks such as editing source code, debugging, testing, deploying and managing code repositories. ODE supports both Oracle and PostgreSQL based on environments. ODE is designed to support the development process for whatever the intent is (Openbravo ERP core contributions, modules and custom code) and whatever the scope is (from small bug fixes to complete new functional modules).

Document explanation

One of the primary goals for ODE is to use open and popular (de-facto standard) tools and practices whenever it is possible. Some of the tools that ODE is based on are Apache Ant, Eclipse IDE and Mercurial. Concepts section introduces these tools which are required to understand how ODE performs. You can skip this section if you are familiar with these tools.

Next section describes Openbravo ERP source code structure: sub-projects within Openbravo ERP source code and folder structure for each of them. After that there is a further explanation about the way Openbravo ERP manages database source code that is a central part of ODE and is done in a very innovative and elegant way.

Next section is a guide addressed to developers about how to perform all the development tasks such as building from sources or committing changes to the repository. All the development tasks are listed and detailed in the Development tasks section.

The next section explains specifics about the three possible development intents: Openbravo ERP core contributions, modules and custom code.

Then you will find a guide to move Openbravo ERP source code from your development environment to a production system.

Finally, the last section describes how to migrate your Openbravo ERP source code to the 2.50 development model.

Concepts

Before starting with the development process let's explain some basic concepts to better understand the rest of the document.

Application Dictionary

See the Application Dictionary section in the Developers Guide.

Source code

It is where all the information required to build the whole Openbravo ERP system is. It includes the Application Dictionary, the Wizard of Application Dictionary (WAD) which is able to generate source code for the elements defined in the Application Dictionary, the code generated by WAD, core classes to manage the system as well as manual code to implement functionality that extends WAD capabilities.

Source code includes Java files which are compiled to generate binary files.

Binaries

During the build process all Java classes in the system are compiled and transformed to binary files. These binary files can be run in a Java Virtual Machine (which is not able to execute Java files directly).

Database

Application Dictionary as well as user's data is stored in a database.

XML model

As Application Dictionary data is part of the Openbravo ERP sources, in order to ease concurrent developments this information can be exported from database to plain XML files. These files maintain all the information required to generate the database and populate it with all data in Application Dictionary.

The synchronization from database to the XML model and vice versa is managed by the DBSourceManager.

Apache Ant

Apache Ant is a Java-based build tool. Openbravo ERP automates most of its development tasks using Apache Ant.

Eclipse IDE

Openbravo ERP source code is Eclipse IDE ready. This means that the setup of Openbravo ERP source code within Eclipse IDE is optimized to be done in a very few steps.

Mercurial

Mercurial is a distributed SCM which facilitates multiple developers to work in the same code managing revisions for each source code file. Openbravo ERP source code is maintained and freely available in the Openbravo source code repositories. So modifications in source code can be done in your local mercurial repository or come from other -external- repositories, this must be taken into account when building the system.

Deploy modes

Openbravo ERP is a web application that runs in a Servlet Container. This means that to build the system it needs to generate the binaries from source code and to deploy the binaries into a Servlet Container, usually Apache-Tomcat. Openbravo ERP build tasks can be configured to manage the deployment in three different modes:

class
class deployment mode copies the compiled Java classes and all necessary files to the Openbravo Context folder within the Servlet Container where the application is served from. Using this deployment mode the application is automatically deployed when the system is built.
war
war deployment mode generates a war file with the whole application. This file can afterwards be deployed on the server. Note that using this deployment mode the application is not automatically deployed when the system is built, but the war file needs to be manually deployed.
none
When Openbravo ERP is compiled all the necessary files to be run in the server are copied to WebContent directory. It is possible to set the server to serve from this directory, which is the standard way to work from Eclipse IDE. In this case Eclipse will manage the deployment.

The deploy mode is set in the Openbravo.properties file, by the deploy.mode property.

Source code structure

Openbravo ERP source code structure is detailed in the Development Project Structure article.

Database code management

Introduction

Openbravo ERP source code is made of two different types of code: MVC code (Java, HTML, xSQL, JavaScript, CSS, ... files) and database code. Database code can be separated into DDL statements of database schema objects (tables, views, triggers,...) and Application Dictionary (AD) metadata (records in AD tables such as AD_Window or AD_Column). When Openbravo ERP is installed, a database (Oracle or PostgreSQL) is created by executing the DDL statements, metadata is inserted and MVC code is compiled. Once the database is created, custom ERP data are added to the database (products, business partners, orders, invoices, ...). Those data are stored by the database in binary files mixed with schema objects and AD metadata.

New developments (bug fixes or new features) usually include changes in both MVC and database code. There are two major problems with the generic exported database dump, especially when wanting to include it in a source code repository:

  1. It is not easy to get a detailed and clean description of your development changes since the dump (*.dmp) file is a binary file and not a text one. Hence, diff statement does not work well on binary files.
  2. It is not possible to update a production environment by deploying the changes in the source code. Instead, a database script needs to be prepared that performs the alter and update statements required in order to keep customer data that is already in the database.

DdlUtils

DdlUtils is a small, easy-to-use component by Apache Foundation for working with Database Definition (DDL) files. These are XML files that contain the definition of the database schema (e.g., tables and columns). These files can be feed into DdlUtils via their corresponding Ant task in order to create or alter the database. In the same way, DdlUtils can generate a DDL file from an existing database. Openbravo has extended some DdlUtils capabilities (for instance, support for check constraints, procedures and views; PL/SQL translation from Oracle to PostgreSQL; support for more database types, ...) and fine tuned others (for instance, export database schema objects, ...) to fully support ODE requirements.

How it works

Each Openbravo ERP system (working copy) has a folder called database where all the database source code (schema objects and AD metadata) are stored in plain XML text files. Source code (plain XML files) inside src/database folder is divided into:

  • model (schema objects such as tables, functions, etc.),
  • sourcedata (AD metadata in tables AD_Table, AD_Window, etc.)

Changes to the database can happen in 2 places:

  1. Inside src-db/database folder (working copy):
    1. Through updates coming from the Mercurial repository. When changes come from a Mercurial update, they do not overwrite changes done in the working copy since these changes are merged within the plain XML files.
  2. Inside the database:
    1. Editing the AD metadata using the Openbravo ERP Application Dictionary windows and processes.
    2. Performing changes to schema objects (tables, columns, procedures and so on) using your database management tool (PGAdmin, SQL Developer, TOAD, etc...).

ODE provides the following tasks to synchronize the database XML files with the database itself:

  • create.database task: first reads schema objects XML files inside src-db/database/model folder and creates schema objects in the Oracle or PostgreSQL database. After that, this task fills these schema objects with AD metadata taken from src-db/database/sourcedata folder. Keep in mind that this task re-creates the database from scratch which means that the existing database will first be dropped!
  • update.database task: compares the schema objects and AD metadata with the database XML files inside the database folder and the database itself. Differences are applied to the database, keeping custom ERP data (products, business partners, orders, invoices, ...) in the database untouched.
  • export.database task: takes schema objects and AD metadata from database and overwrites database XML files inside the src-db/database/model and src-db/database/sourcedata folders, respectively.

As you can imagine, whenever any of these tasks are executed, both database models (the one inside src-db/database folder and the database itself) are forced to be identical. The first two tasks modify the database so that it is equal to src-db/database folder content and the third one overwrites the src-db/database folder content to equal it to the database.

In summary, the src-db/database folder contains Openbravo ERP database source code (plain XML files) clearly separated from custom ERP data (products, business partners, orders, invoices, ...). This way, the database is not distributed as a binary dump file anymore.


Bulbgraph.png   Important note:

Because changes to the database can happen within the text files or the database itself, it is extremely important to guarantee that these changes do not happen simultaneously to both sides because this situation would lead to system inconsistency and loss of data. This is guaranteed by using a check based on the Mercurial revision number. Each time create.database or update.database tasks are launched, revision number of the working copy is saved into the database. export.database task checks that the revision number of the working copy matches with the revision number of the database. If it matches, there is a guarantee that changes in our database will not overwrite changes done by other developers in the database XML files . If it does not, the developer will get an error and will be forced to switch the working copy to the current database revision number.

Different types of data

The default installation of Openbravo will install different data sets resulting in a complete workable system. The previous section already touched on this topic. Within Openbravo we distinguish the following information:

  • sourcedata: this is our application dictionary information. Things like ad_tables, ad_columns, ad_windows go here. This can be extended through modules (modules can add information here), and is updated on each Core update (through update.database). Source data can be found in the src-db/database/sourcedata folder.
  • referenceddata: this is information referenced by sourcedata, like ad_client, ad_org, ad_user, ad_language, or ad_role. The Openbravo database cannot be created without this information (the referencial integrity of the database would be compromised without it), and this explains why it is treated in a special way. It is not updated on each Core update. This data is located in the src-db/database/sourcedata/referenceddata folder.
  • masterdata: this is information which is not needed to create the database, but it's really convenient for every user, like contries, or currencies. This is also not updated on each Core update. This data is located in the referencedata/standard folder.
  • sampledata (smallbazaar, bigbazaar): this is sample information we provide for the users to test openbravo, and contains a new client, with its corresponding business information. This data is located in the referencedata/sampledata folder.

This data is installed/loaded into the database when installing Openbravo.

Development process

File:Build-process1.png

This section explains the most common way of developing Openbravo ERP and which build tasks should be used for each case. In most of the cases it is only necessary to use 3 tasks (install.source, smartbuild and export.database). There are a number of other tasks that can be used but they are not required for the standard process. They are explained in the Development Build Tasks article.

The main task for the standard process is smartbuild which performs an incremental build of the system -only the modified components are rebuilt- as explained below. This task accepts two optional properties: local for local or remote developments which by default is set to yes and restart indicating if after the build process tomcat should be restarted with no as default value.

Local is used as a hint to the build task to let it know if there have been database changes coming from other developers through a pull of the Mercurial repository so it is needed to apply those changes to the database in the local instance. A developer working locally in her/his instances performs all database changes directly in the database so there is no need to update the database to build the system. But if the developer has just done a Mercurial pull then it is likely that other developers have done changes in the XML database files so it is needed to update the database with those changes.

Smartbuild is an incremental process and avoids any task that is not needed. When development is local, smartbuild can skip the update of the database. In any case developers are allowed to update their database from XML files at any time.

Initial installation

After downloading the Openbravo ERP source files (for example from a clone repository using Mercurial) the next step is to install and deploy the system.

First you have to properly configure all the required properties. All of them are stored in the Openbravo.properties file, that you have to configure properly before going ahead.

After all properties are configured, the following step is to build the application from source code and deploy it. All this is done by the install.source task. This task creates the database, inserts sample data on it and compiles and deploys the application accordingly with the deployment mode chosen. To execute it just type in the Openbravo ERP root directory:

<source lang="bash">ant install.source</source>

Local developments

Once Openbravo ERP is up and running it is possible to develop on it. Generally, new developments should be done through modules, further explanations about how to develop modules can be found in the Modularity article.

The standard way for developing locally consists on:

  • Develop/modify windows, tabs, etc through the Application Dictionary.
  • Create/modify database objects directly in database.
  • Develop/modify manual windows editing html, xml, xsql and java files.
Build

Once your changes are done and before you test them it is necessary to build the application. You can do an incremental build by just executing (from command line):

<source lang="bash">ant smartbuild</source>

Remember that by default smartbuild considers only local changes so it does not synchronize the database from the XML files (update database is skipped).

This task generates and compiles the sources for the modified elements, and, depending on the deploy mode, it also deploys them. It is possible to restart tomcat from the same task setting the property restart to yes, this would be:

<source lang="bash">ant smartbuild -Dlocal=yes -Drestart=yes # Note the -Drestart=yes</source>

Database exportation

In most cases developments include modifications in the database. These modifications can be persisted in the database XML files using the DBSourceManager tool. DBSourceManager exports to XML files only the database changes of modules (including core) that are set as In Development. To export the database changes execute:

<source lang="bash">ant export.database</source>

Remote developments

Remote developments are done by other developers remotely and then are merged with the local sources. The main difference with local ones is that remote developments do no modify the database directly. The way a remote development can change objects in database is using XML files, so after updating (merging) the XML files it is necessary also to update the database. After updating the database the process is exactly the same as the local one, this is, compile and deploy the elements that have been modified since last build. All this (update the database, compile last modifications and deploy them) can be done at the same time with the smartbuild command:

<source lang="bash">ant smartbuild -Dlocal=no # Note the -Dlocal=no</source>

The only difference with the local development is in the local parameter which makes the process to update the database in case the XML files were changed.

Validate Database

When a module is exported using the export.database task it is first validated to check for common errors. If the validation fails then the export.database task will also fail and export is not possible.

The following checks are currently done:

  • A table defined in the Application Dictionary should be present in the database and vice versa.
  • Column definitions in the database and the Application Dictionary are compared, any mismatch is reported. The column datatype, default value and length are checked.
  • Tables should have a primary key.
  • Foreign key fields should be part of a foreign key constraint.
  • Names of tables, columns and constraints are checked for their length (Oracle has a 30 character limit there).

You can also run the validate database task directly by executing this Ant task:

<source lang="bash">ant validate.database</source>

This will check the complete database and model. Note that when the validation is done as part of the export.database task then only the tables/columns of modules which are exported are checked.

Validate Module

When a module is packaged with the package.module Ant task, then first it is checked for some common errors. If an error is detected then the package.module task will fail.

Specifically the following checks are done:

  • A module should depend on the Core module, or depend on a module which depends on the Core module (recursively).
  • The path inside the modules/.../src directory must correspond to the Java package defined for the module.
  • The javapackages of the DataPackages which are part of the module must be in line with the Java package defined for the module. So if the module has the Java package org.example, then all Data Packages should have a Java package which starts with org.example.
  • The license type and text must be set.
  • If the module is an Industry Template then it must depend on core and the dependency must be set to 'Included'.
  • If the module adds UI artifacts such as a window or tab then its Translation Required field must be set to yes.

The module validation can be run separately through this Ant task:

<source lang="bash">ant validate.modules -DmoduleJavaPackage=${javapackageofmodule}</source>

Whereby ${javapackageofmodule} equals the Java package of the module.

Test Ant Tasks

Openbravo ERP has a number of Ant tasks for running JUnit test cases. The main one is run.tests: ant run.tests will run the tests which are side effect free. The test Ant tasks assume that the Small Bazaar and Accounting Test clients have been installed.

Development tasks

Development tasks are detailed in the Development Build Tasks article.

Core, modules and customizations

Openbravo ERP is designed to fulfil all customer requirements whatever they are. It is done at different levels each one more specific to the customer than the previous:

  • Openbravo ERP core: common functionality without any industry specific detail that is used for most of the customers.
  • Modules divided in three different types:
    • Functional modules and Plugins: extended functionality (such a HR module) and specific tools or components (such a connection to a remote banking system). More than one module/plugin could be applied to a customer installation.
    • Localization packs: adaptation of Openbravo ERP Core and modules to a country specific operations (translation and specific rules such as chart of account and taxes).
    • Industry Templates: adaptation of Openbravo ERP Core and related modules to a industry specific operations (for example, a vertical for non-profit organizations). Only one industry template can be applied per customer installation.
  • Custom code: customization of Openbravo ERP Core and related modules to completely fulfill customer requirements.
File:FunctionalCoverage.png

So regardless of the scope of a project - just a small bug fix or a large new functional module - development using ODE can have one of the following objectives:

  1. Contribute to Openbravo ERP core,
  2. create a pluggable module or
  3. customize Openbravo ERP to fit some customer requirements.

Customer's production deployment can therefore consist of several elements listed above such as bug fixes, module plugins, new features, etc.

Regardless of the objective, the development process' base is a source code repository updated from different sources. ODE will keep track of the source for each change, but all of them will be managed in the same standard way.

File:DevInt.png

This figure also explains the dependency tree in Openbravo ERP. Openbravo ERP core is completely independent from modules and custom code. A module depends on Openbravo ERP core and other modules it might be based on. Custom code depends on Openbravo ERP core and on all the modules the customer has installed.

The development process is identical for all of the purposes described above.

Core

For contributing to Openbravo ERP Core source code by fixing issues or doing core enhacements, please refer to the contributor's guide.

Modules

For building modules that extend Openbravo ERP Core capabilities, please refer to the modularity documentation.

Customizations

For making customizations that modify Openbravo ERP Core or third-party modules, please also refer to the modularity documentation.

From development to production

Both Openbravo ERP core enhancements and fixes (updates), extension modules and customizations can be packaged into OBX files which can be installed via the Module Management Console (MMC). This new capability makes the jump from a development environment to the production system way much easier.

A software project normally consists of the following environments:

  1. Development environment.
  2. Pre-production system.
  3. Production system.

Development environment

The development environment is where the code deliverables are developed. In Openbravo ERP, functionalities required are coded as explained above and packaged into one or more OBX files, which are the code deliverables.

  • Using Mercurial, each developer can work on its own development environment and easily share the code without having to package it in an OBX file.
  • Once the development is finished, it is strongly recommended that the OBX file generated is installed and tested in different development environments from the ones where it was developed. For installing the OBX file use the Module Management Console.
  • It is also strongly recommended that the database data of the development environments are as close as possible to the production system ones (see Database backup section).

Pre-production system

The pre-production system is where the code deliverables are tested before they are applied into the production system. The pre-production system has to be an exact copy of the production system in order to ensure the success of the final step. This implies having exactly the same Openbravo environment (same components and versions) as the production system, including the same Operating System.

  • Once the development is finished and tested in multiple development environments, just install the OBX file(s) in the pre-production system using the Module management Console and make your tests.
  • It is strongly recommended that the database data of the pre-production system are as close as possible to the production system ones (see Database backup section).
  • You may find useful to apply these performance and security tips to your pre-production system.

Production system

Applying fixes or new developments to the production system is as easy as installing the OBX files that include them using the Module Management Console.

Database backup

Bulbgraph.png   Export of the production system database for its import into the development and pre-production environments is strongly recommended in order to ensure that developments work properly with the freshest production data.
Export

PostgreSQL

To create a dump file of a PostgreSQL database: 

pg_dump -U tad -h localhost -F c -b -v -f /tmp/tad_pgsql.dmp openbravo

In this case the user is tad and the database name is openbravo.

Oracle

To create a dump file of an Oracle database: 

exp tad@XE file=/tmp/tad_ora.dmp log=/tmp/tad_ora.log owner=tad statistics=none

In this case, the user is tad and the SID is XE.

Import

PostgreSQL

First create a new user and a database: 

psql -d postgres -U postgres
CREATE ROLE tad LOGIN PASSWORD 'tad' SUPERUSER CREATEDB CREATEROLE VALID UNTIL 'infinity';
UPDATE pg_authid SET rolcatupdate=true WHERE rolname='tad';
CREATE DATABASE openbravo WITH ENCODING='UTF8' OWNER=tad;
\q

In this case the newly created user is tad with password tad and the database name is openbravo.

Then import the previously exported file: 

pg_restore -i -U tad -d openbravo -h localhost -v -O /tmp/tad_pgsql.dmp

Finally it's recommended to vacuum the database to increase performance: 

vacuumdb -U tad -h localhost -d openbravo -f -z -v

Oracle

First create a new user: 

sqlplus system@XE
create user tad identified by tad default tablespace users temporary tablespace temp;

In this case the newly created user is tad with password tad.

Then import the previously exported file: 

imp tad@XE file=/tmp/tad_ora.dmp log=/tmp/tad_ora.log fromuser=tad touser=tad

Migration from previous releases

The migration (upgrade) of your Openbravo ERP 2.40 production system and development environments is detailed in the How to upgrade your code to 2.50 article.

Languages: 

English  |  Bahasa Indonesia | Translate this article...

{{#hierarchy-bottom:}}

Retrieved from "https://wiki.infinite-erp.co.id/index.php?title=ERP/2.50/Developers_Guide/Concepts/Development_Model&oldid=3887"