Tag: source code management

Source Code Management: Adopting TFVC

This is the second post in a series about source code management. If you haven’t already read the beginning of the story you can find it here.

We’d realised that we’d outgrown our system of one-developer-per-customer and ad-hoc communication between teams about ongoing changes to objects. We needed some more structure and somewhere safe to keep previous versions of objects.

Selecting a System

We evaluated a couple of systems and decided to go with Team Foundation Version Control (TFVC), hosted in Visual Studio Team Services (recently renamed Azure DevOps). The key factors:

  • It’s hosted in the cloud and therefore always available – including for developers who were working on site (something we tend not to do now but happened a lot at the time)
  • Familiar tools – developers were already using Visual Studio for RDLC report development and occasional C# development
  • Source code management is independent of NAV development
    • It doesn’t matter what version of NAV you are developing in as they all support export to/import from text files
    • It doesn’t matter where the NAV database is, you only need to be able to bring the text files back to the local workspace on your laptop
    • We did evaluate a product where previous versions of objects are saved in the NAV development database – but the thought of recreating a development database from a SQL backup and losing the history made it a non-starter
  • We could see there was a lot of scope for future improvements with work item tracking, testing and a comprehensive API
  • We’d already got some in-house experience with Team Foundation Server in our .Net development team

Our philosophy was:

  • We’d have a VSTS project for each customer project
  • A branch in that project to represent each NAV database
  • We’d check-in all vanilla objects at the start of the project and all modifications thereafter

Implementing TFVC

It was pretty hard.

The benefits are an easy sell – we’ll be able to rollback objects, we’ll know what was changed, when, by whom and link to the change request that also tell us why. We could even go live with some changes and not others, in the same file. What’s not to love about that? The practice is a little trickier.

The mindset change was the key thing to try and get right. We were so used to being able to make quick changes to objects. Some consultants were comfortable making minor changes – adding fields to forms and pages for example. Support would also make minor changes and put bug fixes in. Some customers would make their own changes as well.

For source code management to be of any use you have to be able to trust the data. For the data to be trustworthy all code changes must be checked-in, no matter how small and seemingly insignificant.

We didn’t want to put too many obstacles in the way of non-developers customising objects for our customers, after all, most likely the customer bought NAV on the strength of our ability to quickly customise it for them in the first place. However, one way or another, all code changes must make their way to TFVC.

The obvious place to start then was to incorporate TFVC into the development process as smoothly as possible:

  1. Write code in NAV test database
  2. Export object(s) to text
  3. Copy into local workspace and check-in to development branch
  4. Repeat 1-3 as necessary
  5. Merge changeset(s) from test branch to live branch
  6. Download merge changeset(s) from live branch
  7. Import into NAV live database, compile, synchronise (comments about how we ought to have been building fobs and importing them instead can be posted below if you really feel the need)

The main challenge were NAV is concerned is keeping the objects in the database and the contents of the workspace in sync. Fortunately, this was also the time that PowerShell support was being added and we were realising its power for other applications.

For step 2 we allow the developers to export all the relevant objects to a single text file and split them into separate files, one per object, named appropriately into a new folder. Identical to Split-NAVApplicationObjectFile but as a custom function so that we can support forms and dataports.

We also have a PowerShell function for step 6. The developer enters the required changeset ID and PowerShell uses the API to collect all the files modified in that changeset, with contents as at that changeset version, and join them into a single text file to import into the target database. Deleted files are included with a Version List of DELETEME and a message is popped to the developer asking them to manually delete those objects from Object Designer.

Ongoing

Some of the concepts and terms were new to most of us and took a while to get the hang of. Branching, merging, changesets, conflicts, server version, workspace version etc. but nothing that the team can’t get used to with some patience and practice.

Once we were comfortable with the development process and used to crafting sensible, self-contained changesets we had the foundation to start reviewing some code. What a good day that was.

Over time we’ve used the VSTS API and tf command line commands to automate many tasks:

  • Building an environment (database and service tier) for a given TFVC branch
  • Creating deltas of changes between objects in one branch and applying those deltas to objects in another branch
  • Getting the latest version from a given branch of objects that exist in a given folder
  • Creating deltas of the changes between two branches and packaging them into a (v1) extension
  • …a bunch of other cool stuff that isn’t really the point of this post

Suffice to say I can’t imagine us not using source code management now. All of the above only becomes possible once you’re code is checked-in and you can trust its content.

Git

Eventually we reached a point where we started coming up against short-comings of TFVC. We were struggling to use TFVC to accommodate changes in the way that we wanted to work as a team and stared to learn about Git.

I’ll discuss why we decided to move to Git and why it might (or might not) be right for you in the next post…

Source Code Management: A Trilogy in Four Parts

It seems hard to believe now that we ever developed working software without using any source code management system – but we did. For a long time. And judging by the straw polls taken in sessions at NAV conferences lots of partners still do.
In this series of posts I am not intending to dispense any advice based on my own meandering experiences. I’m not Baz Luhrmann. I’ll also try to avoid the slightly self-righteous tone which often seems to accompany advice about source code management and unqualified assertions like, “If you’re using TFVC then you should migrate to Git.”
Instead, I’ll share some of our experiences and reasons for making the decisions about SCM that we did at the time. Hopefully there will be something of use in there for other dev teams at different stages in the same journey.
The stages are these:
  1. Decision to adopt an SCM system
  2. Adopting Team Foundation Version Control
  3. Switching to Git
  4. Conclusions

No Source Code Management?

It isn’t fair to say that we didn’t have any source code management in the old days. We did. The system was composed of certain expectations about how development was done and some manual checks that they were followed.

  • Development was done in the test/dev database before being moved to live
  • You could trust the Modified flag – any objects that had been changed from standard would be ticked as Modified
  • The date and time on the object would represent the last time the object was changed (therefore you could compare object datetimes between databases to judge which objects were different)
    • Developers would often check that the datetime in live was the same as test before starting development in test
  • Dated and initialled comments were added to the documentation trigger and modified lines in the code

In addition to these assumptions, generally

  • The original developer would also put their changes live – they knew which object(s) to move
  • The developer would take a backup of the objects from the live database before importing from test
  • We were a small enough development team that we could talk to each other about changes that had been made
  • The same developer would work on the same customers
  • Support would refer cases back to developers, especially if issues arose after some objects had been moved into live

All of this is, in itself, a system. Sure, there are problems with these assumptions, the steps weren’t always followed and it doesn’t scale too well but this is a source code management system. Of sorts.

Adopting a SCM System

So why did we choose to adopt a new SCM system? The main reasons were these:

  • We’d had instances of SQL databases being deleted which contained the only copy of some work. Lots of hours’ worth of work
  • We often had occasions where it was difficult to piece together what changes had been made and why – particularly if the documentation in the objects was light/absent
    • This was particularly problematic for support developers landed with a case that starts “we’ve had some changes go live recently and now xyz isn’t working”
    • While we’d try to go back to the original developer (see above) if this wasn’t possible the support developer would have the difficult task and trying to work out what had actually changed
  • We didn’t have a reliable, single place to go for previous versions of objects – either to refer or rollback to
    • Developers would take ad-hoc backups of objects before moving them live
    • We’d have a support copy database of most customer’s objects on an internal SQL server
    • We experimented with a homemade backup system that pulled copies of objects from support databases – but not from databases hosted on-premise where development was often done
  • The team was growing – both in terms of staff and number of projects – to the point where it wasn’t practical to rely on the same developer always working on a particular project or on ad-hoc communication between developers, support and consultants to manage the objects

It became clear that we needed a central system to manage our development in. We’d need to:

  • Backup copies of our objects
  • Track changes that had been made, when, by whom
  • Rollback objects to previous versions

Fortunately this realisation coincided with Luc van Vugt running a workshop at NAVTechdays about source code management with TFVC…