Portrait of an IBMi Modernization Project

I cut my teeth on RPG language (the mainstay language of IBMi platform) and experienced application migrations from System/3 on up to IBMi. But, although I’m a staunch advocate of IBMi operating system, over the last ten years I’ve been doing development targeted mostly to Linux-based systems. The truth: I was lured by the sirens of rapid application development heralded by dot-com applications like a social shopping app, K-12 online education, and YouTube video sales and acquisition. Working with the high-speed development style employed by the coders in that volatile industry has been invigorating. Why weren’t they targeting IBMi? Well, these start-ups needed to go cheap. Linux on the cloud is cheap. IBMi is not.

IBMi is a premium solution. It has arguably the computer industry’s best reliability, security, and scalability. If you want to argue about IBMi’s reliability and security, speak with the banking and insurance industry who love IBMi. IBMi scales vertically — that is, on one machine — not horizontally as with Windows and Linux solutions. With Windows and Linux, when you need to grow, you add additional machines — horizontally. With IBMi, a single box can scale — vertically. IBMi also has the industry’s best backup and recovery facilities. Yet, with all IBMi premium features, when comparing large scale applications hosted on Windows or Linux, IBMi has the lowest total cost of ownership. So, yeah, IBMi is not going anywhere anytime soon.

As relevant as the IBMi is, all too often I hear folks say that IBMi is a legacy or otherwise antiquated machine. I’ve heard that recently while doing Grails contracting for a Fortune 100. In fact, my twins, both CPAs with Price Waterhouse Coopers, also call IBMi legacy! They spit that word out like it means old and dying (We at Corgibytes love legacy). They’ve picked up that negative opinion from various Fortune 500 clients that have IBMi in their platform mix. When I pressed my kids on why they have that understanding they responded: Of all platforms at various companies, IBMi was the hardest from which to obtain information. So here’s the thing: the wealth of data kept in the world’s best database and operating system is hidden behind applications that are 20-30 years old.

Everyone knows that an IBMi shop’s biggest problem is application modernization. But know that all the cool stuff I’ve been doing for the venture capital funded dot.com industry can be done on IBMi. I know because I’ve done it. It seems every third or fourth application I do is hosted on or integrated with IBMi. Case in point: kettlerusa.com is an online retail application that I wrote to use DB2 for IBMi and integrate with RPG. I was also on the PowerRuby.com team that brought Ruby and Rails to IBMi.

I’m going to list tools and strategies that a state-of-the-art application development project should be using. Essentially a portrait of the infrastructure of a successful IBMi application start. I’ll start with suggestions on dealing with the daunting task of selecting a language and framework. Then, I’ll recommend tools for source control, testing, editing, collaborative communication, knowledge base management, and project management. And I’ll finish with some considerations for RPG integration strategies and database enhancements.

Language and Frameworks

I’m going to assume you aren’t planning on writing your new application in RPG. You could go with ASNA’s Visual RPG but — even though I was on the original Visual RPG development team — my concern would be finding talent to maintain that code. So, if we limit it to today’s most popular languages, you’ll need to pick from: Ruby, Python, C#, Java, Javascript, PHP, and C++.

Convention over Configuration

Now let’s switch perspective. Instead of picking the language, I strongly urge you to select a framework disregarding the language. Use of a good framework ensures the proper construction and hence maintainability and scalability of your application. Look for a model-view-controller framework that is built on the Convention-over-Configuration (CoC) philosophy. A CoC framework reduces the number of decisions you have to make to provide production quality functionality because the framework defaults to industry conventions. Without CoC — and the benefit of years of research and experience — building similar functionality usually results in a total hack.The first CoC framework was Ruby on Rails (RoR) and, due to its rampant success, a host of Rails-wannabes followed for other languages. Groovy and Grails (GaG) is a prime example. I’ve written well over a dozen GaG applications — one for a Fortune 100.

Want me to give you a quick answer to what framework use? Rails. The Rails community continues to stay ahead of the other frameworks and there’s plenty of websites, training, books, and seminars on Rails.

Tools

All too many developers sit down with a new language and begin to bang out code they expect to eventually be used in a production application. They start by following one or two of the myriad of tutorials for their new language and are encouraged by how quickly they have created a couple of screens. They continue expanding on that initial code thinking it will grow into a production application. That doesn’t work. Those tutorials were written to show the quickest way to learn introductory material in the use of a language. When you keep adding onto that sample code you end up with an unmaintainable mess.

Before you begin development on your new project need to have the following:

• An idea of what you want build

• A source control tool

• A unit testing infrastructure

• A strategy for the acquisition of proficiency of a good program editor

• Configuration of the latest collaborative communication tools

• An application knowledge base

An idea of what you want build

For decades projects were launched with the use of sophisticated project management tools that had innumerable mechanisms for designing the application in copious detail. Many of those products used overblown tools like Gantt charts, UML, and HIPO diagrams. But many great applications have been designed with tools as simple as a stack of 3X5 cards written with flair-tip pens. I suggest you read a few articles and maybe a book on Agile and Scrum but, I’ll tell you, after reading a dozen books and a dozen years experimenting with multiple methodologies, I still think the very thin and easy to read book Extreme Programming Explained: Embrace Change (by Kent Beck and published in 1999) does the best job of describing best practices that you can use without being overwhelmed by process. Kent Beck has recently said that the word Agile has been so overused that now he no longer uses it and tells people to manage projects with agility.

The minimum I want out of a project management system is one that tells me, my teammates, and my boss what I finished, what I’m working on today, and what I might work on next.

The Corgibytes team is preparing a detailed blog post on project management but, until that is published, let me make a couple of suggestions.

First, I’m a true fan of Kanban boards. I originally learned about Kanban in the 1980s when I was a programmer in the manufacturing industry. Toyota invented Kanban in 1953 as a pull-based system that substantially reduced inventory and improved throughput. It turns out that the Kanban philosophy works great for managing IT projects. The simplest form of a Kanban system is just a wall with columns labeled ToDo, Doing, Done and where you slap up sticky notes with tasks scribbled on them. Second, I love the issue management facilities that comes with GitHub (more on GitHub in the next section.) Usage of GitHub’s Issues feature is so pervasive that Kanban snap-ins (HuBoard and Waffle) have become available.

Source control

There should be no development without source control, especially on IBMi. I’ve been pushing source control for IBMi projects in articles since 2002. I first promoted Concurrent Versions System (CVS), as Editor of NetJava Expert, when I asked Barry Kline to write this and then Subversion, in 2005, with this. Subversion was a rewrite of CVS that made many incremental improvements. But 10 years ago Linus Torvalds (the creator of Linux) got fed up with Subversion and designed a source control system from the ground up. The result is an oddly named source control tool called Git.

Having used CVS and then Subversion extensively and, for the most part, being very happy with both — I’ve become a huge advocate of git. And so has everyone else. Companies still using Subversion have it on their radar to move to git.

Use git for your source control. End of story.

You can host a git repository on your IBMi, Windows, Mac or Linux box (or, strangely enough, — in the manner that git was designed — on all of the them at the same time). But I would suggest using github.com. Github.com is free — so long as the project is publicly accessible. If you need your application code to be private, the fees are minimal. GitHub’s $7 a month plan will support most applications. And if your security needs require that your source code be stored only on hardware controlled by you, then you can elect to use GitHub’s Enterprise edition.

Git is crazy feature rich and its command system can be overwhelming. But, in practice, you end up using the same half-dozen commands over and over. And there are GUI clients that make Git very approachable. And there are git integrations for most IDEs, if you useone of those.

Corgibytes will be publishing a blog post on our philosophies and strategies for using Git.

Unit Tests

Just as no application should be without source control: There should be no code without unit tests. M. Scott Ford says “it’s an excellent way to improve productivity and reduce risk” in The Pyramid of Automated Tests. Consider following what I call “test first, code later” or, as the industry calls it, Test Driven Development (TDD). At the very least, slap together “happy path” tests for the questionable or important pieces of code after you’ve written them. Most CoC frameworks come with test infrastructures. Before you write a line of what you expect to be production code, gain an understanding of the testing infrastructures available for your chosen framework and commit to using them. The following is an article I wrote on unit testing with Rails.

We at Corgibytes feel that testing is so important that, in the first phase of bringing in a new Corgibytes client, we review their test code and provide metrics on the coverage of their tests along with recommendations for improving those tests.

The bottom line is: I don’t trust my own code — much less yours — so I’m writing unit tests and I expect you to do so as well.

Programming Editors

Caution: Editor wars ahead! Which is a good thing. Programmers should be fighting about what is the best editor. Even the good folks at Corgibytes can’t agree. Our various developers use Atom, RubyMine, NetBeans, and (my favorite) VIM.

If you are already using IBM Rational Developer for IBMi then you might consider adding a plugin that supports your selected language and framework. But I’d still suggest trying out one of the many other editors that are available.

What I don’t want to see you doing is edit with Notepad. Get a real editor. If you’re actually tempted to use Notepad, then please consider using Notepad++ instead.

Collaborative Communication

Email is dead and no one answers their phone. Well, that’s not completely true but the start of a new application is the time to reconsider collaboration tools. I strongly recommend two: Slack and ScreenHero.

Slack, to over simplify, is a messaging tool. You configure Slack with channels (chat rooms) that are specific to an area. You can identify channels as public or private. Slack (as with most of the tools identified in this blog post) has a free version with plenty of features. Slack keeps a history of all conversations — which is extremely useful for developers. Corgibytes developers are constantly pasting code snippets, directions, and links into Slack. Corgibytes has a channel for each of our clients as well as channels for specific areas like: sales, blogging, standups, and shout-outs. We even have fun channels like: parenting, yoga, and Star-Wars. Personally, I’m too old to be enamored with Star Wars so I’ve never visited that channel — which is an example of why specific channels are handy. Were there a channel called COBOL I wouldn’t have visited it either.

Everybody has used Skype and Google Hangouts. Both have screen sharing capabilities. For 5 years, I paid $50 a month for a GoToMeeting account because it had great screen sharing and allows viewers to take control of the screen. But ScreenHero (which, again, has a free option) is even better than GoToMeeting. Even when developers are in the same office building — because of the shared screen control — I’ve found it to be better to use ScreenHero and have folks stay in their own developer space. I also feel screen sharing sessions encourage more collaboration when all the coders are on their own machine. Screenhero also handles some odd edge cases really well. One good example is keyboard layouts. We have some on the Corgibytes team who don’t use a Qwerty layout, and ScreenHero handles the translations so whoever is typing is able to just type without having to force their brain to use a different keyboard layout.

Knowledge Base

The acquisition of knowledge and understanding is so fluid and transient that all applications must have some form of a knowledge base. Fifteen years ago I knew this and, whenever I was at companies that had no formal knowledge base, I’d install Frank’s Wiki (or friki) in their Java web server. There may be better alternatives than a simple wiki but the thing is: you must have a formal knowledge base. As things are discovered, they need to go into the knowledge base.

I’ve been using the free version of Evernote for three or four years now. Corgibytes used Evernote for a time as well but switched to the wiki feature available with any GitHub repository. I add to our corporate wiki so often that I have it pinned to a tab in my Google Chrome. Also, for all our clients that use GitHub (and most do) we document everything we learn, code, or propose for their applications on their wiki.

Development, Test, and Production Environments

RPG development in the library system of IBMi works well with its library lists. For nouveau development you should have explicitly defined development, test, and production environments. And everything should be optimized for development velocity. Your test database should be transient; it needs to be repopulated on each test to ensure the same context. Note that there are tools integrated into most CoC frameworks that simplify the dynamic construction of database state.

The development database should also be easily rebuildable. It should be easy for anyone to start to work on an application and have a common database context.

Too much trouble? Too much time? I’ve heard and experienced the fallout from that opinion before. Trust me: Pay now or pay later.

I also suggest that your test and development databases be local. Why? Because it improves velocity by reducing latency and removes network security complexities. Now, how’s that going to work, being that you’re targeting DB2 for IBMi? I’ve actually had a lot of luck exporting the DB2 schema and loading it in a local Mac, Linux, or Windows-based database. Yes, I’ve used the various DB2 incarnations, but I usually just use MySQL. It turns out that MySQL supports odd legacy IBMi naming strategies like dollar and pound signs in the field name.

When you test sections of code that indirectly make calls to RPG, write a simple stub. Note, though, you should still have a test that verifies the RPG call integration code functions as expected. Note that I will be publishing subsequent blog posts on strategies for integrating with legacy RPG.

Do and See, See and Do

IBMi’s database and operating system are the world’s best. Help the kids of the world (like my CPA sons) know that. Improve those 20-to-30-year-old applications and databases with a Ruby and Rails or Groovy and Grails or Python and Django or CakePHP application. The beauty of these CoC frameworks — coupled with the use of good tools — is that you can flesh out a web application with one or two tables and a couple RPG calls and quickly have something usable and maintainable. And then you can incrementally grow that application by incrementally adding more features.

I will continue to write IBMi modernization blog posts. Please add a comment if you have any areas you’d like me to cover or if you have additional suggestions for other readers.