I'm talking to QlikTech about their new version, coming soon. I'll be publishing the results in the BI Verdict. They are focusing more and more on these bigger accounts. I think this story fits pretty well in my informal series of posts on the subject of agile BI. By coincidence I have already discussed QlikView in a previous post.
QlikTech has a what they call a "land and expand" policy, which means getting a single department on the tool and expanding from there. Actually, all BI companies that can deliver departmental solutions have something similar. The reason for this is simple: The cost of sales for selling to a company that is already using the tool is much lower than for completely new customers. In fact, a lot of BI tools spread through companies from department to department this way.
Now QlikTech is concentrating more on enterprise accounts. So it's interesting to see that the company is moving away from the previous claim that the tool is a replacement for a data warehouse. I think that any attempt on their part to compete as an enterprise solution would just distract them from their end users.
A lot of BI companies go through a similar life cycle as they grow. Most start out as ways to create departmental solutions, which tend to be faster, more agile projects. As they get bigger management tends to concentrate on larger accounts, which means making sure they are acceptable to the IT department. But IT is more interested in keeping processes running than in agile development. As a result, the products tend to become more complex and less suitable to agile solutions.
This is a big issue for QlikView right now because they have grown so quickly in recent years. But currently the company seems to backing away from radical changes in the tool. But it applies to and BI tool that is growing.
Showing posts with label agile. Show all posts
Showing posts with label agile. Show all posts
Thursday, August 18, 2011
Thursday, July 28, 2011
Why short projects are part of agile business intelligence
One of the key ideas in agility is the importance of delivering real, testable results without delay. In fact, the Agile Manifesto recommends delivering working software frequently, from a couple of weeks to a couple of months.
Delivery working software with two months may sound a bit extreme, but there is good evidence that short projects are more successful than long projects. In fact, our research in the BI Survey shows that the application should be rolled out to the users less than six months after the product has been selected. We have found the same result year after year in the ten year history of the Survey. Amazingly, project benefits start to fall as early as a month after the product is selected, and continue to fall thereafter. And of the many project parameters we study, none shows as clear an effect on project success as project length.
These results from the BI Survey provide clear empirical support for the idea of using agile methods in business intelligence projects. The results have remained also consistent since we started the Survey ten years ago, long before the idea of agile development or agile business intelligence became popular.
But why do short projects work so much better? Our research shows that the main problems that arise in longer projects are organizational, not technical. Needs change over the course of time, and end users lose interest in the project. Disagreements over the project goals arise. Lack of interest and disappointed end users are a major issue in business intelligence.
And needs certainly do change quickly in business intelligence. For example, another study we carried out shows that three quarters of European companies modify their planning processes one or more times a year. In an environment like this, a project that takes a year to implement is quite likely to be obsolete before it is finished. Even a six month wait can push potential users to look around for a more agile solution.
The problem this creates is that not all business intelligence projects can be carried out within a few months. This is especially true when major data management issues need to be addressed. The agile solution to this is to find way of splitting large projects into smaller ones. The usual argument against this approach is that it creates the risk of reducing efficiency in the long term. But the agile methodology is to measure success in terms of working software delivered in the short term, instead of attempting to meet nebulous long-term goals.
Delivery working software with two months may sound a bit extreme, but there is good evidence that short projects are more successful than long projects. In fact, our research in the BI Survey shows that the application should be rolled out to the users less than six months after the product has been selected. We have found the same result year after year in the ten year history of the Survey. Amazingly, project benefits start to fall as early as a month after the product is selected, and continue to fall thereafter. And of the many project parameters we study, none shows as clear an effect on project success as project length.
These results from the BI Survey provide clear empirical support for the idea of using agile methods in business intelligence projects. The results have remained also consistent since we started the Survey ten years ago, long before the idea of agile development or agile business intelligence became popular.
But why do short projects work so much better? Our research shows that the main problems that arise in longer projects are organizational, not technical. Needs change over the course of time, and end users lose interest in the project. Disagreements over the project goals arise. Lack of interest and disappointed end users are a major issue in business intelligence.
And needs certainly do change quickly in business intelligence. For example, another study we carried out shows that three quarters of European companies modify their planning processes one or more times a year. In an environment like this, a project that takes a year to implement is quite likely to be obsolete before it is finished. Even a six month wait can push potential users to look around for a more agile solution.
The problem this creates is that not all business intelligence projects can be carried out within a few months. This is especially true when major data management issues need to be addressed. The agile solution to this is to find way of splitting large projects into smaller ones. The usual argument against this approach is that it creates the risk of reducing efficiency in the long term. But the agile methodology is to measure success in terms of working software delivered in the short term, instead of attempting to meet nebulous long-term goals.
Saturday, July 09, 2011
Discovering hierarchies in columnar databases
I recently blogged about columnar and Wayne Eckerson asked me for a clearer explanation of what I mean by columnar databases "discovering hierarchies".
For example consider the approach of two well known products, IBM Cognos TM1, which is multidimensional, and QlikView, which is columnar.
My definition of a data model is a structure that is informed by an administrator, or set down in the master data. To me this is different to a structure derived from analyzing the transactions. In the following simple example, let's say I have two sales teams, one for dental hygiene products and one for soap.
If I were designing a data model in TM1, then I could create a hierarchy, which is a set of parent child relationships between the departments and the products they sell. If the soap people cross-sold some toothpaste, it would have no effect on the hierarchy, because it is predetermined by my idea of how my company is supposed to work.
If I were to import the same data in QlikView I could create a report that showed me the relationship between the sales teams and the products without defining the model. Once the data is imported, QlikView recognizes the relationships automatically.
When the soap guys cross-sell toothpaste, QlikView discovrs the new relationship, but the hierarchies stay the same in TM1, because that's how I defined the model. To me this is the key difference. On the one hand the structures are coming directly from the actuals, and on the other hand they reflect my predefined perception (or "model") of what is going on.
So columnar databases typically discover the relationships automatically, and multidimensional databases allows you to define the relationships as you want them. Another way to look at this is that the transactional data drives the master data structure in a colunmar database, but those structures are wired into the multidimensional model.
So which approach is better? It depends on the application.
For example consider the approach of two well known products, IBM Cognos TM1, which is multidimensional, and QlikView, which is columnar.
My definition of a data model is a structure that is informed by an administrator, or set down in the master data. To me this is different to a structure derived from analyzing the transactions. In the following simple example, let's say I have two sales teams, one for dental hygiene products and one for soap.
If I were designing a data model in TM1, then I could create a hierarchy, which is a set of parent child relationships between the departments and the products they sell. If the soap people cross-sold some toothpaste, it would have no effect on the hierarchy, because it is predetermined by my idea of how my company is supposed to work.
If I were to import the same data in QlikView I could create a report that showed me the relationship between the sales teams and the products without defining the model. Once the data is imported, QlikView recognizes the relationships automatically.
When the soap guys cross-sell toothpaste, QlikView discovrs the new relationship, but the hierarchies stay the same in TM1, because that's how I defined the model. To me this is the key difference. On the one hand the structures are coming directly from the actuals, and on the other hand they reflect my predefined perception (or "model") of what is going on.
So columnar databases typically discover the relationships automatically, and multidimensional databases allows you to define the relationships as you want them. Another way to look at this is that the transactional data drives the master data structure in a colunmar database, but those structures are wired into the multidimensional model.
So which approach is better? It depends on the application.
Subscribe to:
Posts (Atom)