Study: What are IS’ main issues?


What are the major problems architects, developers and consultants face when building Information Systems (IS)? We received 52 responses from 34 different companies. Here is what they shared with us.

We were interested in the exciting issues faced by IS departments on a daily basis and wanted to know which were the most important to address.

The questionnaire, dataset and a link to the raw data are provided at the end of the article.

What are the three most significant problems?

In our study, we asked the participants to choose the three most significant problems they face from the following list:

  1. Keeping abreast of how the IS works
  2. Identifying where to position a new feature
  3. Designing a new feature and analyzing its effects
  4. Developing a new feature in a reasonable amount of time
  5. Identifying and describing operating anomalies
  6. Correcting anomalies
  7. Preventing regression when adding new features
  8. Technologically updating the IS

The two most significant responses were the challenges of keeping abreast of how the IS works and technologically updating the IS (both of these answers had been experienced by 60% of respondents). In third place was the challenge of developing a new feature in a reasonable amount of time (40%). This is huge, not least given that IT teams dedicate a significant amount of their time to this very task.

Last place, with less than 7% of the vote, was taken by the challenges of identifying where to position a new feature and of correcting anomalies.

Let’s take a look at the three most important problems in detail.

1. The challenge of keeping abreast of how the IS works

Let’s talk about IS professionals’ first challenge, and according to the study, they all experienced the same kinds of obstacles :

The highest-voted obstacle, reported by 78% of participants, is incomplete or outdated documentation. This is surprising, in that this problem persists despite there being plenty of information available. So, is documentation really something that should get pushed further and further down our “to do” lists? Are the benefits of proper documentation misunderstood or minimized? Are developers complacent? Do organizations allocate overly tight budgets in this area? To address this, we could, for example, use a proxy which automatically generates documentation for an API, further centralize architectural documentation, implement the use of standard documents to help guide developers or introduce a reward for regular “documenters” such as a mega-millions bonus. (Actually, on second thoughts, forget that last one.)

The second most troublesome obstacle is the sheer number of people involved (65%), followed by information being held by external agents (40%) or distributed throughout the organization in such a way that makes it difficult to access (37%). These problems highlight the importance of reinternalizing information and making it accessible, and of empowering teams to reacquaint themselves with the IS.

2. The challenge of technologically updating the IS

The challenge of technologically updating the IS was respondents’ second most important problem. What are the obstacles?

57% of participants said that they were impeded by the coupling between systems. Coupling is the degree of interdependence between a client and a server in a specific version. The client refers to the system which relies on the other - the “server” – so that it can function. The server itself is independent and does not need the client to function. The higher the coupling is, the more code there is to edit when the server changes version and the more difficult it is to replace it with another. This is the problem that Semantic REST APIs help us to address. For example, they enable servers to update themselves without having to modify clients.

The next obstacle after coupling, with 50% of the vote, is the fear of breaking a feature due to very low test coverage. Testing is something we all get drilled into us during our studies and, while they are sometimes considered annoying and time-consuming to write, tests ultimately bring us peace of mind. In this situation, Test-Driven Development is our ally. This method encourages developers to take test-writing as their starting point.

Finally, lengthy roll-out processes seem to be the last major obstacle. While we understand very well that deployment should be a non-event and that there are companies that do this well, such as Amazon and Netflix, others continue to use processes which can take up to six months. This seems like a particularly important point to address given that, in the new economy, a company’s ability to bring new services and products to market is a key development factor.

Differing views based on profession

It is interesting to note the difference between how architects and developers experience these issues. The architects complained the least about coupling, with only 33% remarking on it as compared to the average of 60%. As for the fear of breaking features, architects placed this in their top three worries, with 65% of them voting for it as compared to just 37% of developers. This can be explained by the fact that architects design the interfaces which initiate the coupling, while developers develop the features and write the tests. We naturally tend to feel that the biggest problems are those associated with tasks we are not personally responsible for, otherwise we would have found a solution for them, right?

3. The challenge of developing a new feature in a reasonable amount of time

We asked those surveyed to use a scale from 0 to 5 to rate their ability to develop new features in a reasonable amount of time. We deliberately kept the meaning of “a reasonable amount of time” ambiguous so that each participant could apply the phrase to their own work. It is impossible to define what a “reasonable” amount of time should be across all projects and companies. Half of the participants rated their ability at 3/5, and the average score was 2.5. We consider this rather low, bearing in mind the issue’s centrality to many teams of computer scientists’ day-to-day work.

On average, participants who are required to use technologies found that they slowed them down, giving a score of 3/5. The following are three ways of improving the situation:

  • (i) enable the IS department to learn about new technologies more quickly,
  • (ii) test new technologies on isolated projects and provide formal feedback, and
  • (iii) offer training and discussion groups around the technologies covered by the IS department.

As to the three factors which most negatively affected their ability to develop new features in a reasonable amount of time, 58% said that the specifications provided to them are not sufficiently informative; 54% are slowed down by frequent changes in management; and 46% are slowed down by work procedures. Once again, we are faced with organizational and managerial factors.

Different visions in different professions

When we examine the responses for each profession in greater detail, one thing jumps out: each profession is very much aware of the issues associated with their tasks and the resources allocated to them. Consultants called to answer specific questions and not to track day-to-day developments are twice as likely to mention managerial and procedural issues. Architects are most concerned by the technologies they are required to use because they impede their primary job. Fewer architects voted for the issue of having less-informative specifications than developers did. This is probably because they have a more active influence over specifications, while developers are dependent on them.

From this, we deduced that when we closely examine people’s daily tasks and the tools provided to each of them, the feedback becomes even further divided. In this area of work, it is crucial to consult people from each profession in order to gain an objective and comprehensive view of the issues that need to be addressed. Architects are neither the voice nor the representative of developers. They simply see the situation differently.


Of the ten most important obstacles reported, six are organizational, three are procedural (e.g. documentation, specifications and tests) and one is technical (i.e. coupling between the systems).

This highlights the importance of addressing the subject of Semantic REST APIs, in that these provide a solution to the main technical problem associated with IS. While many people write about organizational problems, technical problems are rarely examined in depth, and yet technical coupling is an important subject! It significantly increases the time to market, which is strategically essential in the digital age. That is why we have dedicated this series of articles to this exciting subject. Together we will learn how Semantic REST APIs, which combine hypermedia and linked data APIs, can help us. Get ready to learn about this subject or to refresh your knowledge. We will give you some ideas to get you started.

If organizational issues interest you, there are many publications which address them online, in the media and on this blog. We dedicated a previous series to new types of organization, which you can find here.

See you soon at the next episod of our series.


Details about the dataset

The questionnaire was sent out between December 4, 2018 and January 20, 2019. The raw data results and analysis are available on GitHub. Of the 52 participants, 10 are architects, 9 are developers, 18 are both architects and developers and 12 are consultants. The analysis was conducted category by category based on the following data:

  1. All participants’ responses to the question under analysis,
  2. Responses weighted by profession (i.e. we balanced the results so that each profession had the same weight, as if we had received the same number of responses from developers, consultants, etc.), and
  3. Responses sorted by profession.

The percentages reported in this article are weighted.

The participants work in 34 different companies, both French and international, of which two-thirds have more than 1,000 employees.

We also wanted to analyze the results based on the size of the participants’ IS. To do this, we asked them to report how many virtual machines there are in their IS. Unfortunately, the distribution of responses was significantly unbalanced, with 10/52 unable to respond, so we ruled this out as a possibility.