How To Display Too Much Data
We're building a web-based platform where the main dashboard shows a table of data for users to view and analyze. As we're growing this tool, we seem to be adding more and more columns and are running out of horizontal space on smaller screen resolutions (we're seeing some data wrap). It is important to the user to view all columns at once, so column hiding is somewhat out of the question, as is horizontal scrolling (per the boss's request and because horizontal scrolling sucks anyways). I was wondering if anyone knew of a clever UI for tables with many columns.
Below is a screen capture of the layout of the table. Unfortunately for security purposes I had to blur out the column titles. This example has rows with practically no data in them, but in use they will have lots of data that will expand wider than their respective column titles. In response to some of the answers, it is imperative that the user see all the columns, as they are all necessary to analyze the data to make certain decisions. All the data is numeric aside from the first three columns. Thanks again for any feedback.
Ok so I see that uploading this image doesn't really do it justice... it takes up the full width of the screen at 1680x1050 resolution (22") and as data fills in the columns, the left column w/the text shrinks down a bit.
EDIT 2 I just wanted to say thank you to everyone's awesome ideas. I can only select one answer, but a lot of your ideas have proven very useful and may end up in the final product. Thanks again!
If the data doesn't have to be tabular, build graphs or visualizations that show multiple benchmarks at a quick glance. That, or completely rethink what's 'important'. Without knowing more, it's hard to give any more feedback.
I also think you should give more information or, better, provide some visual aid. With what we know now, I don't see many alternatives.
@greelmo, the data DOES have to be tabular and it includes "sparkline" graphs in the data cells already in addition to the capability to show larger graphs on any existing data set. @rahul, @pam i will see if i can get a SS
Do you really need to see ALL of the columns at once? I understand you might need to compare column A with columns B, F, R, and Z. I can also understand if certain columns (particularly the first three) might need to be visible at all times. But do you really need to compare ANY given column with ANY other column?
It frustrates me when people ask for things like this as if it is perfectly reasonable to fit 2,000 cubic feet of cargo into a minivan.
Why not just have a constant stream of binary digits scroll down screen. There are no rows but related data items are color coded in shades of green. Then your boss just needs to put on his "Matrix" sunglasses.
copy windows explorer ( where you view files and folders ) like multi rows per row, rounding , etc.
Another frustrating thing about this question is that one of the key reasons/constraints isn't listed. In a comment to another answer, you wrote, "the user is looking intensely at a lot of data that is changing in near real time [as he edits]". This gives a key reason for both having the monster view and for editing right there in the table, though there may of course still be better ways to fulfill both contraints.
Hiding and not hiding are two ends of a continuum; e.g. a numeric column could collapse into a clickable sliver of color / skinny icon. Lie Ryan's comments about sorting are right on and using sorting (to effectively hide rows instead of columns?) is very helpful. But if the sort order doesn't/can't show recently changed rows first, then it could help to have a real-time highlight appear on the last N cells that have changed, or on cells that have changed in the last M minutes. Maybe it would even make sense to auto-expand minified columns if anything in them (that's on-screen?) changes.
@JonCoombs - my thoughts are similar. Once it was clarified *why* all the columns needed to be visible at once [real-time changes], then the natural solution is to make sure that anything changing is seen. Any sane solution would also find some way to use multiple rows per left-hand item. To not interfere with easy visual comparison up/down between rows, this could be done by having a second table showing the columns that did not fit in the first table. E.g. mark 3 or 4 rows you are comparing, extra info for those rows only show up in a table below. Even better, *delta* values in red/black +-
Since all of the data after the first three columns is numeric, you can round off so that the width of each column is as little as four characters.
Full Length Shorter Tiny ------------ ------- ---- $143,573.39 $143.6k 144k 112.54% 112.5% 113 198,220,329 198.2m 198m
You might decide which version to display based on the amount of space available. Those fortunate to have larger monitors would still get everything. The presentation would degrade gracefully for smaller monitors
Of course, when the contents of a cell are abbreviated, you should show the full value in a tool tip.
Update: @Oskar pointed out in the comments that showing numbers with different magnitudes makes it more difficult to compare visually. So here's a variation in case comparing relative values is more important than precision.
Revenue ($millions) ------------ 143.6 93.2 0.3 < 0.1
(The header in this example is wider than the data, but as others pointed out that can be solved by tilting the headers at a 45 degree angle.)
this is the type of answer i was looking for. i talked to my boss about this and he likes this idea. thanks! this has definitely pointed me in the correct direction.
While this helps you *cheat* squeezing in data it IS NOT the correct solution. Please go the extra mile to really understand what exactly the end-user is doing at this screen. I'd be willing to bet that either a) no, the user really doesn't need to see all those columns or b) no, the user really doesn't need to see all those rows.
@xanadont if you were to make that bet, you would lose. please believe me when i tell you that EVERY PIECE OF DATA ON THAT SCREEN IS EXTREMELY IMPORTANT.
@xanadont i do understand with great detail what my end user is doing, which is why i am asking this question on how to show them more data
If it's so important perhaps the default approach would be computer analysis of the data instead of human analysis?
When L2 added this it looked a bit neat but.. also a lot harder to size numbers against eachother at a glance... they "fixed" that by adding different colors depending on if it was raw values, K's or M's and so forth...
You can try displaying multiple lines per record, e.g.:
You can also go a step further by making each row a summary view that can be expanded so that the user can see the complete record, e.g.:
In the pic above, the user is interested in Mary Jane's complete info, so she clicks on the triangle next to her name to see the complete record.
Also, I disagree with your premise that "column hiding is somewhat out of the question". If there really is that much information, the user is going to want to hide something. You may want to have them all visible by default initially; but there should be a way for the user to hide some columns as she goes.
column hiding is out of the question because the boss says so (believe me, this was my first suggestion). your answer is a good idea, however. the only problem i foresee is that we don't really have text data, but pretty much all numeric data that is dynamically totaled on the client meaning there's really nothing to expand on.
What kind of reason for anything is "because the boss says so"? Why does he say so?
@rahul if the person who signs your check says to do something, you do it. i presented column hiding to him and he said firmly no. he said no because he wants to see all the columns all the time.
@Jason no offense, but I wouldn't just do what my boss says if he ordered me to do something wih no reason; I'd get a different job. If you're working for this guy (as a UI designer?), why does he not let you do your job? Arbitrary orders like what you're describing don't suggest an atmosphere of trust to me.
I find that when I approach my superiors with humility and ask them to explain their reasoning (as a student would ask a teacher) my questions tend to be well received.
@rahul i didn't ask a question on here on how i should talk to my boss about something. i have a problem with certain constraints that i am to work within and was wondering if anyone had any ideas on a creative solution. telling me to try to change the constraints isn't what i'm looking for. it's like telling a person dying of thirst in the desert to just go buy a bottle of water :)
If the columns are out-growing the rows, you can try to put the rows as columns
ha! very creative... won't work for my application (mostly numeric data) but an interesting solution nonetheless :) thanks!
It could work for You, but requires a huge change and I bet You don't want to have to rewrite ;) IMHO it's the best idea here. Vertical scrolling is good.
@naugter it is definitely a good idea, but doesn't work for comparing numeric data. and yes, it would be a bitch to rewrite.
I'm not sure I understand how you can't compare data in columns just as easily as data in rows.
Since horizontal scrolling was already excluded this can't be a solution unless you've only got enough rows(now columns) to fill the screen width. In a standard table you can scroll vertically to compare unlimited rows without having to adjust where you're looking. The boss wants "all the columns, all the time" and will presumably be using vertical scrolling for such numeric comparisons.
This sounds a lot like the case where the customer/product manager wants everything on the main screen vs. everything you need on the main screen.
IMHO, a dashboard screen should give the user an overview of everything... but to get details on a portion of that (e.g. the 275% increase in sales this week) the user should "drill-down" into the data to see the table(s) of info on where the sales came from.
Over-complicating the "main" view to try and show lots and lots of information ends up failing as the user gets lost in a swath of data. The examples below are just random things I found on Google, but in screen A you can hardly even decide where to start - vs. Screen B where content is neatly organized into overviews.
example of overwhelming data:
example of simple, easy to digest data:
+1. The insight here is that the customer doesn't necessarily know what she wants or what's best in a given situation. That's the job of the interaction designer! Trust us. We know what we're doing. ;)
Thanks for this answer, but I think you are overassuming here. When I say that we need to see all the columns, I really mean that we need to see all the columns. Also, by dashboard, I didn't mean overview... I guess I wrote that because that's what we call it here internally. It's really a table of data that the user must see and modify inline. Every column is necessary, which makes the problem with horizontal space that much of a problem.
Jason, it's hard for us to help you when you give reasons like "because that's the way it is". If you could try explaining *why* you need to see all the columns, we might be able to come up with some other ideas. The fact that scunliffe's answer is getting upvoted suggests the community feels that this is the best answer based on the information you've given us.
@rahul the "because that's the way it is" response is only because there are some areas that i have no flexibility, such as whether or not all the columns are visible. the reason that all the columns must be visible is because the user needs to know the data in all those columns in order to update (or not update) statuses and payments in real time. every bit of data is crucial for this decision, and in the future, there will be more columns as well. it seems to be apparent that there's no real way to deal with this problem aside from simply limiting columns, unfortunately.
@Jason - I feel your pain, but from what you've noted in the comments and the screenshot you added, it appears that you are essentially rendering a "spreadsheet" on screen. There's nothing wrong with a spreadsheet, if that is what the user *really* needs, but just like in MS Excel - you will only be able to get 12-15 meaningful columns on screen and to see the rest you *will need* to scroll horizontally. My advice would be to reduce each column's header to provide enough info, but not overly exceed the width of the data in that column. (see next comment)...
@Jason (cont'd) - E.g. if you have a column that will only contain a single character (e.g. Y/N/?, or 0/1/2/3/? or T/F) if you can use an icon or abbreviated text in the header you may be able to greatly reduce the column width. e.g. if you have a "Number" column, using "#" instead will save some space.
@scunliffe i had this feeling as well. re: using chars as column headers and a key, i also thought about this and presented it as an option and was shot down. good idea, nonetheless :) thanks so much for your input
@Jason The user really *must* edit data inline? Yeah? NO! The user should not be editing data in a table. Sorry, that's just plain, bad design. The designers are thinking like programmers and that's precisely why you're in this predicament. Editing the record should be pulled out into a details view. Oh, but the table lets you see other records. First the users probably don't need to see the other records. But, if they really do, allow multiply opened details views, cloning, and / or other functions. Anything to get that damned table out of my view.
@xanadont i disagree with what you think is bad design. while it may be bad design for some applications, for this application it is absolutely necessary. the user is looking intensely at a lot of data that is changing in near real time. for him to be whisked away to another screen to make a tiny change and then sent back again would be bad design. also, his changes are relevant to all the other data around him, so pulling out into a details view is distracting.
+1, I really like the idea of the dashboard, especially the idea of separated concerns in panels.
A detail view doesn't have to appear to "whisk you away" - that's a design problem easily overcome.
Table is a good way to present large amount of data. However, with more than 5 columns, tables quickly become unreadable. If your data is changing in real time as you say it, the user most likely won't be able to make his decision in time if he had to look at 15 different columns at the same time.
Google knows best that the most important consideration when looking at a large amount of data is sorting. You need to sort by its relevance to user. Every piece of data may be important, but some piece of data are more important than the others, you need to identify which piece is the most important and which pieces are not, this heavily depends on the business' goal.
Not all columns are created equal
Some columns are more important than the other:
- some columns must be read first before data in another column becomes relevant. English speaking users reads from left-to-right, columns should be read left-to-right when possible;
- some columns are important only in their relative order, i.e. their absolute values don't really matter, e.g. date-time. You can hide this column, and use sorting to give the sense of order;
- some values are only important as high, medium, low. The small difference between two "high" values may not really affect decision-making by much since there are other factors that affects decision more significantly. You can use "green", "yellow", "red" icon instead of the full text to save space.
- some columns may be read-only, or need to be edited less often than some other, there are groups of columns that are very likely to be edited together. Sort your columns so that the columns that requires editing together is grouped together;
Not all rows are created equal
A very common mistake when sorting, is to sort by some useless arbitrary order, like alphabetic order or date or numerical values. Ordering that is static and consistent is useful if you need to search values, but you should let the computer do the searching for you. Instead, in most cases you'd want to order by its relevance.
Some of these may or may not apply in your case:
- recently updated data are more important;
- higher values transactions or transactions in a certain range of values are more important;
- false values or true values are not important;
- (more complicated) false values are important only when another field has a certain value, otherwise it's true value that is important
You need to identify these goals for your users, and and do a weighted sorting of your data and columns accordingly. If you have not identified which goals are the most important for your users, then you're not really designing. After you identify which data is most important, then you can proceed to hiding some of the less important datas, or abbreviating them, or abstracting them into icons. Icons are easy to absorb, and your user don't want to spend three seconds reading a 8221 and 1463 if he can spend one-tenth of second looking at "green circle" and "red cross".
Some values are important only in certain context
Highlight interesting values. The user may only be interested in transactions with a certain value in a certain field. And the user might know he'd never touch transactions which is too high or too low for his preference; you can filter them so the user does not even need to see them. You can even hide a whole column, if the user say he doesn't have a use for the column.
Excellent, excellent answer. One small thing I'd tweak would be to try a "minified" column instead of a hidden one, since that appears to be anathema to this customer. That might be a non-informative but clickable thick separator line, or something slightly wider that shows a color/icon (and also expands when clicked).
what data does the user actually need to see?
If only we could get a straight answer to that. Instead of "it depends".
I keep on running into this problem, and every time I Google I end up back here at Jason's excellent question.
And there are many excellent answers, but I smell a meta-topic that we all seem to skirt around yet I think is worth unpacking.
First get a handle on the true range of User Intentions
When faced with situations like this, I find it useful to take a step back and ensure we have a good appreciation for the diversity of user intentions within our system. I drew a simple diagram to illustrate what I mean:
Here I'm considering two things:
- How well can we define the users' activities?
- How well can we predict when users need something?
If we know that every morning Sales Managers review their sales pipeline by value and close date, then we can design an dashboard for that and know it should be "close at hand" every morning. This is what I'd call a defined and pervasive use case.
When we see investment analysts pouring over the stocks in their portfolio, we know they are constantly assessing their positions using a range of standard techniques, but we can't pin down at any given moment exactly how they are doing the assessment. This is especially true if it is in response to an external stimulus (like a call from a colleague on the trading floor, or a story in the Financial Times). This is what we might call undefined yet pervasive.
Perhaps we know that department heads are responsible for managing their costs and budgets. But do we understand what managing exactly entails? And this may change over time - for example at the start of the year the company could be focused on market share and revenue optimisation, while by Q4 everyone is focused on cost reduction. This can lead to very different kinds of review and action. This scenario I might call undefined and ephemeral
And so on .. I think you get the picture.
Deciding Where to Focus
The upper-right quadrant is obviously the best place to be as a designer. We can pin down what users really need, and put our effort into designing the best experience for them. We can eliminate all the cruft in our application that doesn't add value to the experience. We can win awards if we do it well;-)
Unfortunately, the world is a messy place. Chances are that our users are all over the chart. And that can change from minute to minute.
Our usual response, and still the best advice you will hear regardless of discipline (programmers, designers, entrepreneurs) is simply focus. In extremis: do one thing and do it well.
If you are in a startup or working on an MVP, this is easy and self-selecting ("we only build this for users who want to do X").
For everyone else, I'm suggesting we instead need to appreciate first which quadrant(s) users want and need to work within, and use that as a lens to focus our effort. If our users work in Quadrant D, but we are trying to force them into a Quadrant A solution .. it will not be pretty, and the users will always win in the end anyway.
The real Jason probably long moved on, but many of us have our days when we are Jason. From his description, it seems the starting assumption/proposition is that Jason's users are all about Quadrant C - always very busy doing stuff we can't easily define with the data sets we provide.
In other words, classic knowledge workers. We particularly struggle to build systems for these folks, because they know and respond to lots of things that our poor computers haven't a clue about.
It is no surprise that the humble spreadsheet still rules supreme in places like finance departments. Once the ERP systems are done with all the transaction processing, most of the work remaining falls squarely outside the top-right Quadrant... and a spreadsheet is still the ultimate blank canvas for working with data.
So what can we do? I'd suggest two parallel strategies
Identify use cases for Quadrant A treatment
OK, maybe most of the work needs to get done in the pseudo-spreadsheet view. But maybe not all. When we pick up indications of defined-pervasive usage, we can build specifically for that (another view, dashboard page, or punch-out).
There are many good answers and comments already attached to this question that are in a way getting to this: how to find the Quadrant A gem in the mess of Quadrant C?
How do we find gems in the rough?
- old school - spend hours/days/weeks working with users to find out what they really do
- find champions from the user community who can explain how things really get done
- look at the data. Can we discover predictive behaviours leading up to a defined action? e.g. before placing a sell order, users always seem to look at views A and B, sort by C and D, and run report E. It may at least give us a place to start examining more closely...
Build the best Quadrant C experience possible
But with systems like this, it is often a Taylorist pipe-dream to think we can eventually reduce it all down to a neat set of use cases we can design Quadrant A solutions around.
If we can't build for complete and well-defined end goals, perhaps we can still make great leaps by focusing improving the journey for the users?
Focus on Access and Affordances
So we don't know exactly what users are trying to do, but we can improve their ability to access and manipulate our data. For example:
- best-in-class searching, filtering and sorting
- column show/hide. Better yet: show/hide column groups
- easy download of data to do my own analysis
- select from a set of pre-defined views for specific purposes
- maybe I can save and use my own custom views? Maybe I can share that view with co-workers
- as a user I can "scale" my view as required. For example:
- font size
- more/less detail per cell
Focus on Micro-Tooling
We don't know the whole story of what you are trying to achieve, but we can deliver parts of the puzzle. For example:
- obvious drill-down on rows for more info
- "right-click" tools:
- pivot by this column
- goal-seek a new value for this cell
- pop-up historical trend for this row/cell
- pin this row and rank others around it
Focus on adding value to Data
We don't know exactly how our people are turning our data into Information-Knowledge-Wisdom, but we should be looking for opportunities to continually improve the value we can add to the data.
e.g. today I just give you raw production numbers, tomorrow I can give you that plus the change since yesterday/last week/last month.
So while people will probably still be in control of extracting knowledge and wisdom, perhaps we can get better at presenting information instead of just data.
( Jason lives! ) This is a fantastic articulation of the problem. I'm addressing this in a reporting product right now and it's really exciting to see the Quadrant C users getting into a sticky experience. It can be done!
Very nice answer. "column show/hide. Better yet: show/hide column groups." Even better still (IMHO): minify/expand columns or column groups. Collapsing/expanding is much less scary and easier to manipulate as a user. It could be a clickable, 3-pixel wide separator, or add a few more pixels and show a color or skinny icon. Clicking expands to the actual number.
+1 just for the chart alone, even if the rest of your post were completely horrible advice (which I'm sure it isn't)
I know you've selected an answer, but I wanted to bring up something that I think is pretty important: data vs. information. I've been thinking a lot about this recently, so it's at the front of my headmeats.
What you're doing is showing data. It's raw, it all seems really important, but in reality most people don't need to see the data. What they really need is information that's been generated based on the data. You mention in one of the comments that users can "see a graph when they hover over the row", and I think that's backward. The raw data should be available on demand, but the default view should analyze the data and show the resulting information.
This isn't to say graphs are the correct choice, but an alternate display of the key findings of all that data might be a wiser way to go. This is harder to develop, conceptually, but will make the end product more of a useful tool than simply a place to lookup things. The question to ask your users is "what do you want to know when you're looking at this screen?"
+1 process the data (refine it) in a way that matches what human reviewers already do with the data. The ultimate top-level view would be a nice RED-YELLOW-GREEN scorecard for each section and the possibility to drill down on domain items and rules which created the score...
What we do is divide the screen basically in two parts. The upper half shows details about the currently selected item, the lower half shows the grid with a paged list of items available.
The tab controls are used to display detail/ collection views of what we call nested details/ collections. These are formed by foreign key relationships.
Hope this helps you somewhat.
One example I can think of that might inspire something is MLB.com's standings page, which is a table with way too many columns, but the customization interface is instantly intuitive (to me - and it's not aimed at a techie audience). You probably don't want MLBAM's arbitrary limit on the maximum number of columns you can have, of course.
The biggest UI problem with column hiding is usually getting the hidden column back - unless they've fixed it recently, "unhide" is really hard to find in Excel, for example.
+1 this is also the type of answer i was looking for. this is a very creative solution that i will present. thanks for this link!
i wish i could accept two answers, because this one is great. it's gotten preliminary approval by other devs and should probably be approved by el jefe. thanks so much!
I'm a bit surprised, that none has suggested the Master-Detail pattern yet.
We have exactly the same problem in several of our own products: Too many columns to fit on a screen, however, every piece of information will be relevant in some use-case.
Actually, we decided to do, what you(r boss) ruled out: Cutting the default number of visible columns to roughly 5-6 carefully selected attributes to cover 90% of use-cases. For the selected row however, we display all attributes in a property pane on the right side of the screen. The user sees the most important attributes of as many rows as possible in the table, and as many attributes as possible for the selected row in the property pane.
The user may still add/remove columns from the table, just like in the Windows Explorer, if she deems other columns more important than the ones choosen by us, or even dismiss the property pane and resort to horizontal scrolling, if the user decides that's what she really wants.
The introduction of the property pane has some nifty advantages:
- You can provide more meaningful editing UI if the data is meant to be edited
- Often, you can present the data more appropriately than in a table cell. Think of an photo taken of an employee, a set of GPS-coordinates shown on a map, the possibilities are endless.
Pardon the labels being German, but it should get the idea across: