Migrating our internal system from Notes

Stanislav Marszalek  31 August 2016 14:03:48
As it seems that Notes are no longer developed by IBM, some time ago we’ve started the preparation for the migration of our internal system from Notes and today we’ve finished the final one month of test period and go to full production regime, while switching off Notes apps.

We had several applications in Notes for our use, like:

  • Contact
  • Projects
  • Activities
  • Activity Report
  • Absence Reports
  • Travel Expenses
  • Pricelist
  • Inventory
Image:Migrating our internal system from Notes

They included simple forms, workflows, email notifications, some kind logic on save, approval or close. We wanted to move the entire functionality to the new system not only moving the data for reading or simple editing.

As we’ve started to move some our clients to Vaadin and MongoDb, obviously we’ve chosen that platform for our system as well, to prove that it works correctly in real life applications and it is useable at least in the same manner as in Notes client. The project was split up into those parts:

1.        Data migration – we use our export to Mongo tool. It was a matter of linking of the fields on both sides. Some of them as it is, some of them converted by formula to the different type (string to boolean, integer to string etc) and some of them changed from simple type into the object. Special attention required readers and authors fields which were joined into one field in MongoDb for clarity. That all required several exports back and forth to tune everything it up.

2.        Application migration – that was the toughest part, because all codes from QueryOpen, QuerySave, agents, actions have to be rewritten together with forms and views design. But there is no other way if you want to work in the same way as in Notes previously. In that process some of unused parts were removed but some new things were included into application. Saying that, that is the good chance to modernize your applications. It takes around 2 months to convert above mentioned applications into Vaadin.

3.        Testing – there were two stages here. We moved the data for the first time and started to test the basic functionality. In the same time our Notes applications run still in the production. After 2 weeks of testing we stopped Notes completely, restrict the access for editing to all, moved the data to the new system and run it as production in test mode, which means that all bugs which prevent the application from basic functionality were fixed ASAP. But after previous testing there were just few such bugs.

So we can say that the migration was not without the problem but on the other hand it was much more straightforward then we’ve expected. What are our main observations from users regarding new system:

  • It is very similar to Notes one. Form layouts are the same, the functionality is the same, and therefore no training is needed for new applications.
  • Search and overall response of the system is much faster than in Notes.
  • Search can be done across more databases and it can search for word variation, incompleteness etc.
  • It can include variety of web elements, like charts, summary panels, calendars etc. which are not available for Notes client.
To see, how such converted from Notes application can look like, just go here using:

login: john.hill@demo.com
password: demo

         

   

 

Vaadin - the platform of choice for Notes developers - JavaScripts, Agents and DEMO - IV. part

Stanislav Marszalek  3 May 2016 10:47:46
In the first part of our series I've focused on overall reasons why we've chosen Vaadin as a next development platform for our future applications. The second part shows how to navigate and work with views and in the third part we went through variety of form components. Today we set the attention to Java Script components, agents and access rights. At the end is a link to our simple demo in Vaadin.

JavaScript Components

As mentioned earlier programming in Vaadin means that everything is done in Java on the server. There is no need to write HTML, Javascripts, call Ajax. The code in Java can interact with client side, so your code can have a function which gets a document from the database, then displays confirmation message box to the user, wait on his decision and processed with the backend operations. To do this in standard web application (in Xpages for example) you need to involve not only backend classes like Java or PHP, but also Javascripts and Ajax calls.

But there are a bunch of very useful Javascript libraries out there, which would be very nice to use in Vaadin applications. For example Full Calendar, Google charts, Canvas3D and others. Good news is, that there is a way how to do this. Here is detailed description. Basically you need to connect client-side with server-site in both directions:

·        RPC calls from client to server

·        Shared state and RPC from server to client

We did it, for example to Full Calendar. Java code fetches the events from the database and send it to Full Calendar for the display. On the other hand, when clicking on the day in Full calendar, user fires the request which is captured by Java and that displays the event dialog form draw by Vaadin. In the same way you can implement whatever JavaScript library into your application.

Image:Vaadin - the platform of choice for Notes developers - JavaScripts, Agents and DEMO - IV. part


Celý článek "Vaadin - the platform of choice for Notes developers - JavaScripts, Agents and DEMO - IV. part" »

Vaadin - we take platform seriously ... so we passed certification ...

David Marko  28 April 2016 07:45:00
Image:Vaadin - we take platform seriously ... so we passed certification ...

Vaadin - the platform of choice for Notes developers - Forms - III. part

Stanislav Marszalek  8 April 2016 15:46:16
In the first part of our series I've focused on overall reasons why we've chosen Vaadin as a next development platform for our future applications and the second part shows you how to navigate and work with views. Today we want to show form components. Like in Notes you can use lot of different form components which allow the user to enter the data.

TabSheets

Most of our Notes forms are built with tabs to separate the form sections. Usually there is basic tab, history tab, ACL tab and many more depended on the form purpose. There is TabSheet In Vaadin component which can do the same for you. Similar to that is Accordion except that the "tabs" are arranged vertically
Image:Vaadin - the platform of choice for Notes developers - Forms - III. part


Celý článek "Vaadin - the platform of choice for Notes developers - Forms - III. part" »

Vaadin - the platform of choice for Notes developers - Navigation, Views - II. part

Stanislav Marszalek  1 April 2016 13:07:03
In the first part of our series I've focused on overall reasons why we've chosen Vaadin as a next development platform for our future applications. Utilizing our almost 20 years of experience with Notes applications we’ve tried to bring “the best” of Notes platform to new web apps. So here is how we replace the main components of apps.

Desktop look and Navigation

We get quite good response from users about the basic navigation in Notes apps. By this I mean the left navigation panel, from which you can open the content to the right panel using tabs to display multiply content. The same we try to replicate to web Vaadin applications. Fortunately Vaadin includes so called layouts elements which you can use for drawing the components like views, forms, and navigations. You can see an example below, there is no HTML code behind and everything is created from Java classes.

Image:Vaadin - the platform of choice for Notes developers - Navigation, Views - II. part


Celý článek "Vaadin - the platform of choice for Notes developers - Navigation, Views - II. part" »

Vaadin – the platform of choice for Notes developers – I. part

Stanislav Marszalek  29 March 2016 13:24:30
Our company develops Notes applications for years and it is still our main source of income, but several years ago we’ve started to see a gradual decline in the demand of applications based on Notes. And looking around, this seems to be a problem not only in the Czech Republic, where we are based, but also in other parts of the world. The reason is obvious, it started with pure IBM marketing followed by insufficient technical improvements of the product and IBM’s move into different areas like social software.

Image:Vaadin – the platform of choice for Notes developers – I. part

All of this forced us to rethink our future development some two years ago. We’ve started with seeking the data storage, similar to Notes database. After lot of testing we chosen MongoDB as main data store and Elastic for searching. MongoDB can store various data, together with attachments, there is no real limitation in size and number of data and also here you can achieve access control to each document, but in that case you have to program it. There is no real Notes replication from client to the server also but nowadays web applications are not based on that in most of the cases.

To make Notes data transformation even easier, we’ve developed a tool for data export from Notes to MongoDB in shape of Notes application. Here are more details about it.

Application platform was the second choice. We started with Symfony PHP framework. It is very good framework which allows us to build very powerful web applications and we are able to do the same thing as in Notes application, including export do Word, Excel, PDF, send emails, set up access rights to the document and much more. Here is more info about it. The problem is quite steep learning curve, because PHP was entirely new system to us and it takes a time to learn it properly.

Image:Vaadin – the platform of choice for Notes developers – I. part

In the meantime we’ve began looking for some Java based platform, because Java is the main language for Xpages application and Xpages are still half of our Notes development program nowadays and therefore our developers are more familiar with Java. That is why we’ve started with Vaadin, which has many advantages. Also blog posts from Rene Winkelmeyer, Sven Hasselbach, Paul Withers and others, whose connected Vaadin with Notes, were encouraging.

Image:Vaadin – the platform of choice for Notes developers – I. part

So in this and following blog posts, I would like to show you Vaadin platform and our way of implementation of various features, which are quite familiar from Notes also.

The reasons why we chose Vaadin as a main development platform for the future were this:

  • Vaadin is Java based so if you are doing Xpages development, the language is the same. You can still develop Vaadin application and then easily switch to Xpages development. It was not so with PHP.
  • Vaadin can work with Notes data directly so you can use it instead Xpages. Although our primary datastore is MongoDB, because if we want to offer our application to new client, there is no fee for Notes user licences and that makes the application significantly cheaper.
  • Entire application in Vaadin can be written only in Java! There is no HTML coding, there is no JavaScript and that makes the development more straightforward and cleaner. When you develop in Xpages, you sometime don’t know, if that element is getting here from JavaScript or is it from @formula or from SSJS. In Vaadin everything is in one source code, in Java classes.
  • Vaadin makes application more “business like”. In standard web application it is not obvious to scroll down in view using arrows, to open document by Ctrl+E, to save document using Ctrl+ S etc. but in Vaadin you can do this easily and that is very similar to Notes client behaviour.
  • Vaadin applications are fast! Not only because it uses MongoDB behind but also the load of elements like forms or views are very fast and this is the first user feeling when switched from similar Notes application.
In the next blog post we want to focus on how specific elements, like views, picklists, dialogboxes, familiar from Notes client, can be built in Vaadin.


For those on Connect 2016: Have you seen plans for the next Notes release?

Stanislav Marszalek  2 February 2016 13:51:37
Hi guys.
we develop an applications on Xpages and one of the big issue here, is the old Java version in Domino. IBM promises to upgrade Java in the next release, but we still don't know when this will be. Has anybody mentioned more accurate release date then just the half of some year?
Thanks a lot for comments and have a good time in Orlando!

Notes data on Android mobile ...and offline

Stanislav Marszalek  5 January 2016 10:44:16
How to get data from your Notes database to Android mobiles and use them offline? That was a task which we need to solve in recent days. Users should be able to edit and save their contacts, activities on Android mobiles and then send them back to Notes database. But not only texts, numbers, dates but also images, audio and files. And that all in offline mode as well. On top of that, the application should access mobile’s sensors like location, speed, camera etc. The last point means that we have to create native Android application, so we’ve chosen Android Studio to do the job but how to store data on mobile.
Image:Notes data on Android mobile ...and offline

Where to store data on mobile

You can pick up from several options, how to store your data on mobiles. You can use text files or more obvious choice is SQLite database, in-memory mobile database for storing structured data. But we need to synchronize that data with server, which means write your own code which will handle all synchronization challenges, something like Notes replication and that is not quite easy. That is why we’ve selected Couchbase database which promises to solve that issue.      

Couchbase mobile and Sync Gateway

Couchbase database is like Lotus Notes, no structured database, which stores its data in JSON format. Apart from standard server, it has also mobile version, which can be installed on mobile together with your application. Mobile application stores data in Couchbase Mobile and then replicate them with server part using Sync Gateway. Main replication features:
  • Push and pull replication
  • Replication can start on command or it can be continuous. It means that replication is active whenever user is connected to the internet. For instance, user saves his contact and that change is replicated to the server within seconds.
  • Replication includes not only primitive data but also images, files.
Using Couchbase infrastructure we get data from mobile devices to the Couchbase server, but we still need to move them to Notes databases.
Image:Notes data on Android mobile ...and offline
Java synchronization agent between Couchbase and Notes

For the synchronization purpose we’ve created Java agent in Notes database, which compares modification dates on both sides then compares values of each record and creates, updates or deletes records on specific side. It was not as tricky as one can image. Especially replication conflicts can be avoided by setting up proper permissions, assign which device has higher priority, shortly by set up specific application logic. For Java agent development we’ve use Couchbase Java SDK. Here is a snapshot of Java code:
Image:Notes data on Android mobile ...and offline

    Change your Domino into SQL

    Stanislav Marszalek  26 November 2015 13:06:46
    We bring you a presentation from SUTOL conference in which we focused on creating dynamic charts from Notes data using AlaSQL javascript library. Do you need to group your Notes data according your criteria? Do you want to filter out specific documents? Do you like to see records within the time range you specify? Basically, do you want to send SQL commands to your Notes data and don’t let users wait forever? Then come have a look on this presentation, in which I show you how to build up a drill-down reports connected to Google Charts in XPages using AlaSQL.js library. AlaSQL.js is in-memory SQL database which is very fast and can work with JSON arrays and objects to which you can convert your Notes data.

    Here a presentation:
    Image:Change your Domino into SQL2

    SUTOL Conference - About Notes in Prague - 11.11.2015

    Stanislav Marszalek  5 November 2015 05:48:00
    Image:About Notes in Prague - 11.11.2015 SUTOL Conference

    We would like to invite you to SUTOL Conference in Prague where we will have a presentation about using SQL-like databases in Xpages.
    The 7th SUTOL Technical Conference will be held on November 11, 2015 in Prague. For the first time, the agenda is full in English and up to 200 attendees is expected mainly from Central and Eastern Europe.
    SUTOL, The Czech "Lotus user group", runs two conferences per year - one business oriented and one technical - with a stable audience of 100 people. This year, we decided to make the autumn event more international to bring together customers, business partners, sponsors and IBMers across country borders. The conference has two tracks (administration and development) focused on IBM Collaboration Solutions portfolio - IBM Notes/Domino, IBM Connections, IBM Sametime, IBM Connections Cloud and others.

    Here is an abstract from our session:
    Boost your Notes database with SQL-like features - Stanislav Marszalek
    Do you need to group your Notes data according your criteria? Do you want to filter out specific documents? Do you like to see records within the time range you specify? Basically, do you want to send SQL commands to your Notes data and don’t let users wait forever? Then come to our session, in which we will show you how to build up a drill-down reports connected to Google Charts in XPages using AlaSQL.js library. AlaSQL.js is in-memory SQL database which is very fast and can work with JSON arrays and objects to which you can convert your Notes data. In this session we will show you a technical overview of the entire process, user interface and a glimpse of the code behind.

    Here is the link to the conference details.
    TOPlist