Languages:

Model Driven Development (MDD)

InfiniteERP follows a Model Driven Development (MDD) approach. This means that InfiniteERP uses a technology agnostic model to define application components, such as windows and processes. Based on this application model, code and other software artifacts are generated.

InfiniteERP data model information — so called metadata — is stored in the InfiniteERP Application Dictionary.

Model Driven Development aims to increase productivity and re-use through separation of concerns and abstraction. The model is an abstract definition of system components which contains enough information to drive the generation of one (or more) implementations of the system in a concrete technology.

This separation of concerns — abstract functional description in the model and implementation of model components in a concrete technology — hides technology complexities to ERP domain experts in their process to define and implement new ERP functionality and simplifies the evolution of the implementation technology.

In some cases it is needed to code a solution externally from the model. This is fully supported by InfiniteERP. Developers can freely develop their own solutions on top of the InfiniteERP technology stack.

Development Environment

InfiniteERP developers have three different ways to develop their code. Following the MDD approach, most common is to edit InfiniteERP Application Dictionary through a web browser connected to InfiniteERP. Based on the new model definition the software artifacts can be generated automatically. A developer can also connect directly to the InfiniteERP database through a SQL client (e.g. pgAdmin, SQL Developer) to manage database schema objects (tables, procedures, etc.). Finally developers can develop their own code through an integrated development environment such as Eclipse or IntelliJ IDEA.

All InfiniteERP software artifacts are stored in text files in the development project. This includes the database definition and content. The large advantage of using text files for storing all software artifacts is that it is much easier to share and compare changes made by developers in a distributed environment.

InfiniteERP uses a tool called DBSourceManager to manage database source code. DBSourceManager is able to read from the database schema objects and application dictionary data and export them to XML files. It can also create or update an InfiniteERP database from those XML files.

The process to build the system from InfiniteERP source code includes a number of steps to generate the code at different levels (DAL, WAD and others) and put together that code with other code directly written by developers. InfiniteERP has automated this process through build tasks.

Concepts

Before starting with the development process, let's explain some basic general concepts.

Database Model

It contains the structure of all the database elements used in InfiniteERP: tables, triggers, views, sequences and functions. The Database Model is part of the XML model.

Application Dictionary

The Application Dictionary is the part of the source code of InfiniteERP stored in the database. It is a set of declarative definitions of business elements and logic that are used to build and render the application. For example it contains the definition of windows, tables, forms, reports, processes, etc. It is stored in AD tables such as AD_Window or AD_Column and is part of the XML model.

XML Model

The XML model contains the database schema, the Database Model and the Application Dictionary. In order to ease concurrent developments this information can be exported from database to plain XML files. These files maintain all the information required to generate the database and populate it with all data in Application Dictionary. It is stored in a neutral language compatible with the two database engines supported by InfiniteERP: PostgreSQL and Oracle.

The synchronization from database to the XML model and vice versa is managed by the DBSourceManager.

Source Code

It is where all the information required to build the whole InfiniteERP system is. It includes the Database Model, the Application Dictionary, generated entity classes, core classes to manage the system as well as java code, javascript code, html, reports and other files to implement functionality that complements the Application Dictionary.

Binaries

During the build process, all Java classes in the system are compiled and transformed to binary files. These binary files can be packaged and deployed to a J2EE application server like Tomcat.

Database

The Database is where the Application Dictionary and user's data are stored.

Build Tool

InfiniteERP automates most of its development and compilation tasks using a build tool (Ant-based build tasks in the Openbravo CE lineage).

IDE

InfiniteERP source code is IDE-ready. This means that the setup of InfiniteERP source code within an integrated development environment (Eclipse / IntelliJ IDEA) is optimized to be done in very few steps.

Source Code Management (SCM)

A distributed SCM (Git/Mercurial) facilitates multiple developers to work in the same code managing revisions for each source code file. InfiniteERP source code and packages are maintained in source code repositories.

Deploy Modes

InfiniteERP is a web application that runs in a Servlet Container. This means that, to build the system, it needs to generate the binaries from source code and to deploy the binaries into a Servlet Container, usually Apache Tomcat. InfiniteERP build tasks can be configured to manage the deployment in three different modes:

class — copies the compiled Java classes and all necessary files to the InfiniteERP Context folder within the Servlet Container where the application is served from. Using this deployment mode the application is automatically deployed when the system is built.
war — generates a war file with the whole application. This file can afterwards be deployed on the server. Note that using this deployment mode the application is not automatically deployed when the system is built; the war file needs to be manually deployed.
none — when InfiniteERP is compiled, all the necessary files to be run in the server are copied to the WebContent directory. It is possible to set the server to serve from this directory, which is the standard way to work from the Eclipse IDE. In this case Eclipse will manage the deployment.

Source Code Structure

The source code of InfiniteERP is structured in different folders:

config — all the configuration files of InfiniteERP. The most important files are the build properties files that contain all the deployment options and database connection properties.
legal — the InfiniteERP licence and the licence files of all the third party components used by InfiniteERP.
lib — all the library files, separated between lib/build (libraries needed only to perform build tasks) and lib/runtime (libraries needed in runtime and during build tasks).
modules — all the modules installed or in development, with all their artifacts, each in its own subfolder.
referencedata — the standard and sample data stored in XML files.
src — the main folder for all the core source code files and resources.
src-core — source files and resources of the openbravo-core.jar library.
src-trl — source files and resources of the openbravo-trl.jar library.
src-wad — source files and resources of the openbravo-wad.jar library.
src-util — used for the Build Validation, Module Scripts and Diagnostic components needed to perform maintenance tasks for InfiniteERP and modules.
src-db — source files and resources of the DBSourceManager tool and also the files of the Database Model and Application Dictionary.
src-gen — all the DAL entity generated java classes.
src-test — all the source files and resources of the tests.
web — all static web files like javascript files, images, skins, etc.
WebContent — where all the files needed to deploy InfiniteERP are copied when executing the building tasks. An application server can serve the InfiniteERP application directly from here or a package file can be built from here to be deployed.

Database Code Management

Overview

InfiniteERP source code is made of two different types of code: source code (Java, JavaScript, CSS, HTML files) and database code. Database code can be separated into DDL statements of Database Model (tables, triggers, views, sequences and functions) and Application Dictionary.

When InfiniteERP is installed, a database (Oracle or PostgreSQL) is created by executing the DDL statements, application dictionary data is inserted and code is generated. Once the database is created, custom ERP data are added to the database (products, business partners, orders, invoices, etc). Those data are stored by the database in binary files mixed with the Database Model and the Application Dictionary.

New developments (bug fixes or new features) usually include changes in both source code and database code. There are two major problems with the generic exported database dump, especially when wanting to include it in a source code repository:

It is not easy to get a detailed and clean description of your development changes since the dump (*.dmp) file is a binary file and not a text one. Hence, diff statement does not work well on binary files.
It is not possible to update a production environment by deploying the changes in the source code. Instead, a database script needs to be prepared that performs the alter and update statements required in order to keep customer data that is already in the database.

DBSourceManager

DBSourceManager is based on DdlUtils, a small, easy-to-use component by Apache Foundation for working with Database Definition (DDL) files. These are XML files that contain the definition of the database schema (e.g., tables and columns). These files can be fed into DBSourceManager via their corresponding build task in order to create or alter the database. In the same way, DBSourceManager can generate a DDL file from an existing database.

InfiniteERP has extended several DdlUtils capabilities (for instance, support for check constraints, procedures and views; PL/SQL translation from Oracle to PostgreSQL; support for more database types, etc.) and fine tuned others (for instance, export database schema objects) to fully support the development environment requirements.

How It Works

Each InfiniteERP system (working copy) has a folder (src-db/database) where all the database code (Database Model and Application Dictionary) is stored in plain XML text files. Source code in plain XML files is divided into:

model — Database model.
sourcedata — Application dictionary.

Changes to the database can happen in 2 places:

Inside the src-db/database folder (working copy): through updates coming from the SCM repository. When changes come from a repository update, they do not overwrite changes done in the working copy since these changes are merged within the plain XML files.
Inside the database: editing the Application Dictionary using the InfiniteERP Application Dictionary windows and processes, or performing changes in the Database model (tables, procedures, etc.) using your favourite database management tool (pgAdmin, SQL Developer, TOAD, etc.).

The development environment provides the following tasks to synchronize the database XML files with the database itself:

create.database task — first reads the Database Model XML files inside the src-db/database/model folder and creates schema objects in the Oracle or PostgreSQL database. After that, this task fills the database with the Application Dictionary taken from the src-db/database/sourcedata folder.
Warning: keep in mind that this task re-creates the database from scratch, which means that the existing database will first be dropped.
update.database task — compares the Database Model and the Application Dictionary stored in the database with the XML files inside the src-db/database/model and src-db/database/sourcedata folders. Differences are applied to the database, keeping custom ERP data (products, business partners, orders, invoices, etc.) in the database untouched.
export.database task — takes the Database Model and Application Dictionary stored in the database and overwrites the XML files inside the src-db/database/model and src-db/database/sourcedata folders.

Whenever any of these tasks are executed, both models (the one inside src-db/database folder and the database itself) are forced to be identical. The first two tasks modify the database so that it is equal to the src-db/database folder content, and the third one overwrites the src-db/database folder content to equal it to the database.

In summary, the src-db/database folder contains InfiniteERP database source code (plain XML files) clearly separated from custom ERP data (products, business partners, orders, invoices, etc.). This way, the database is not distributed as a binary dump file anymore.

Important: since changes to the database can happen within the text files or the database itself, it is extremely important to guarantee that these changes do not happen simultaneously on both sides, because this situation would lead to system inconsistency and loss of data. This is guaranteed by using a check based on the SCM revision number. Each time create.database or update.database tasks are launched, the revision number of the working copy is saved into the database. The export.database task checks that the revision number of the working copy matches the revision number of the database. If it matches, there is a guarantee that changes in the database will not overwrite changes done by other developers in the database XML files. If it does not, the developer will get an error and will be forced to switch the working copy to the current database revision number.

Different Types of Data

The default installation of InfiniteERP will install different data sets resulting in a complete workable system. Within InfiniteERP, we distinguish the following information:

sourcedata — the Application Dictionary information. Data of tables ad_tables, ad_columns, ad_windows go here. This can be extended through modules (modules can add information here), and is updated on each Core update (through update.database). Application Dictionary data can be found in the src-db/database/sourcedata folder.
referencedata — information referenced by sourcedata: data of tables ad_client, ad_org, ad_user, ad_language, or ad_role. The InfiniteERP database cannot be created without this information (the referential integrity of the database would be compromised without it), which explains why it is treated in a special way. It is not updated on each Core update. This data is located in the src-db/database/sourcedata/referencedata folder.
standard — information which is not needed to create the database, but is really convenient for every user, like document types, document templates, etc. This is also not updated on each Core update. This data is located in the referencedata/standard folder.
sampledata — sample information provided for users to test InfiniteERP, containing a new client with its corresponding business information. This data is located in the referencedata/sampledata folder.

This data is installed/loaded into the database when installing InfiniteERP.

Development Process

This section explains the most common way of developing InfiniteERP and which build tasks should be used for each case. In most cases it is only necessary to use 3 tasks (install.source, smartbuild and export.database). There are a number of other tasks that can be used but they are not required for the standard process.

The main task for the standard process is smartbuild, which performs an incremental build of the system — only the modified components are rebuilt. This task accepts two optional properties: local for local or remote developments (default yes) and restart indicating if after the build process Tomcat should be restarted (default no).

Local is used as a hint to the build task to let it know if there have been database changes coming from other developers through a pull of the source code repository, so it is needed to apply those changes to the database in the local instance. A developer working locally in her/his instance performs all database changes directly in the database, so there is no need to update the database to build the system. But if the developer has just done a repository pull, then it is likely that other developers have done changes in the XML database files, so it is needed to update the database with those changes.

Smartbuild is an incremental process and avoids any task that is not needed. When development is local, smartbuild can skip the update of the database. In any case, developers are allowed to update their database from XML files at any time.

Initial Installation

After downloading the InfiniteERP source files (for example from a clone repository), the next step is to install and deploy the system.

First, you have to properly configure all the required properties — all of them are stored in the build properties file, that you have to configure properly before going ahead.

After all properties are configured, the following step is to build the application from source code and deploy it. All this is done by the install.source task. This task creates the database, inserts sample data on it, and compiles and deploys the application accordingly with the deployment mode chosen. To execute it, just type in the InfiniteERP root directory:

ant install.source

Local Developments

Once InfiniteERP is up and running, it is possible to develop on it. Generally, new developments should be done through modules.

The standard way for developing locally consists of:

Develop/modify windows, tabs, etc. through the Application Dictionary.
Create/modify database objects directly in the database.
Develop/modify windows editing html, xml, javascript and java files.

Build

Once your changes are done and before you test them, it is necessary to build the application. You can do an incremental build by just executing (from command line):

ant smartbuild

Note: by default smartbuild considers only local changes, so it does not synchronize the database from the XML files (update database is skipped).

This task generates and compiles the sources for the modified elements, and, depending on the deploy mode, it also deploys them. It is possible to restart Tomcat from the same task setting the property restart to yes:

ant smartbuild -Dlocal=yes -Drestart=yes

Database Exportation

In most cases, developments include modifications in the database. These modifications can be persisted in the database XML files using the DBSourceManager tool. DBSourceManager exports to XML files only the database changes of modules (including core) that are set as In Development. To export the database changes execute:

ant export.database

Remote Developments

Remote developments are done by other developers remotely and then merged with the local sources. The main difference with local ones is that remote developments do not modify the database directly. The way a remote development can change objects in the database is using XML files, so after updating (merging) the XML files it is necessary also to update the database. After updating the database the process is exactly the same as the local one — compile and deploy the elements that have been modified since last build. All this (update the database, compile last modifications and deploy them) can be done at the same time with the smartbuild command:

ant smartbuild -Dlocal=no

The only difference with the local development is in the local parameter, which makes the process update the database in case the XML files were changed.

Validate Database

When a module is exported using the export.database task, it is first validated to check for common errors. If the validation fails, then the export.database task will also fail and export is not possible.

The following checks are currently done:

A table defined in the Application Dictionary should be present in the database and vice versa.
Column definitions in the database and the Application Dictionary are compared; any mismatch is reported. The column datatype, default value and length are checked.
Tables should have a primary key.
Foreign key fields should be part of a foreign key constraint.
Names of tables, columns and constraints are checked for their length (Oracle has a 30 character limit there).

Test Tasks

InfiniteERP has a number of build tasks for running JUnit test cases:

ant test

These tasks will run all tests defined in InfiniteERP and installed modules.

Core, Modules and Customizations

InfiniteERP is designed to fulfil all customer requirements whatever they are. It is done at different levels, each one more specific to the customer than the previous:

InfiniteERP core — common functionality without any industry specific detail that is used for most of the customers.
Modules, divided in three different types:
- Functional modules and Plugins — extended functionality (such as an HR module) and specific tools or components (such as a connection to a remote banking system). More than one module/plugin can be applied to a customer installation.
- Localization packs — adaptation of InfiniteERP Core and modules to country specific operations (translation and specific rules such as chart of accounts and taxes).
- Industry Templates — adaptation of InfiniteERP Core and related modules to industry specific operations (for example, a vertical for non-profit organizations). Only one industry template can be applied per customer installation.
Custom code — customization of InfiniteERP Core and related modules to completely fulfil customer requirements.

So, regardless of the scope of a project — just a small bug fix or a large new functional module — development can be divided in one of the following categories:

Core — a modification of the source code provided by the InfiniteERP distribution.
Module — a pluggable module that can be packaged independently from the core of InfiniteERP, distributed and deployed to other InfiniteERP installations.
Customization — to fit some customer requirements, sometimes it is needed to update the InfiniteERP core source code that cannot be packaged in a module.

Customer's production deployment can therefore consist of several elements listed above such as bug fixes, module plugins, new features, etc.

Regardless of the objective, the development process' base is a source code repository updated from different sources. The development environment keeps track of the source for each change, but all of them are managed in the same standard way.

The dependency tree in InfiniteERP: InfiniteERP core is completely independent from modules and custom code. A module depends on InfiniteERP core and other modules it might be based on. Custom code depends on InfiniteERP core and on all the modules the customer has installed.

The development process is identical for all of the categories described above.

References

This page is a derivative of Main Development Concepts and Development Model by Openbravo Wiki, via the Etendo Documentation (Etendo and InfiniteERP are both successors of Openbravo), used under CC BY-SA 2.5 ES. This work is licensed under CC BY-SA 2.5.

Model Driven Development

Contents