Being able to understand how your product is being used is the most important part of building great product for a startup. Monitoring and measure the right things from an admin dashboard is a great way to learn. Once you’ve started letting users in, you’ll want to split your time between quantitative feedback (cohort metrics / analytics) with qualitative feedback (surveys, activity streams and interviews). Having an admin panel will help you accomplish this.

That being said, choosing the right metrics and making decisions based on that data still falls into the artist side of being and entrepreneur.

Object metrics

I’m going to use the object metrics as a reference to the core “thing” your product offers. For us at Flowtown, or core object is a gift. So in the descriptions below when I reference Object ID’s, etc I’m referencing your products focused “thing”. In our situation every account creates gifts and sends them out. The screenshot above outlines some of the key metrics and activities that we monitor to understand how users are using our application and in what order. We use this information to look for patterns regarding super active accounts and potentially extract customer archetypes that we can then iterate against.

When you first start building out your metrics you don’t need to have it 100% instrumented, just enough to learn and allow you to make decisions that will move your product development forward in a meaningful way. For most companies the core metrics should be retention – do your users comeback to your app? In alignment with that goal, you’re object metrics should educate you on when they returned, what they did and how that impacted their experience – were they successful?.

Capturing The Data

We use a combination of Google Analytics and internal activity logs (system events we store for all our users in our product). Google analytics and their API makes its extremely simple to instrument our front end pages and then correlate that with our internal metrics by matching up our Object ID’s.

Once you’ve captured the data then you’ll want to report on it in some fashion. I break these screens into 2 types of reports: Product (Educates you on the product), and Non-Product (System, data, tools, etc) reports.

Product Focused Reports

I believe in building “getto-but-useful” admin reports. Nothing fancy but informational. Ideally you should have an account search that lets you look up any account by email or name and shows you all their activity. Each object should also have this information view so you can see how other entities are reacting against it (ex: If you have a gift, who claimed it?). In our product customers give gifts, and recipients are allowed to re-gift them to friends – so it’s important for us to know when those actions occured and if the recipients are successful in claiming their gift.

Metrics dashboard: In our office we have a monitor hanging on the wall (it actually looks like it’s floating) that lists system metrics and our core (at most 3) product metrics that we’re actively focused on for that month. We choose to use Geckoboard since they have a pre-designed front end and the integration is super simple. I’ve also update my default tab on my browser to the dashboard URL so I can review it first thing in the morning.

Activity streams: This is a real time running log of all the activity (events) in our system. We then color code certain actions that we’re interested in learning more about. This essentially functions as a heart beat of the product and quickly shows us where users get stuck and how they use our product. It’s invaluable.

Cohort metrics: These reports are super simple and always focused around key metrics (retention, activation and feature usage). The goal of this report is to know if we’re getting better over time and reflecting on previous cohorts as our baseline. If our activation dips from 60% to 40% the following week, we quickly realize we messed something up. Using that time stamp (week period) we can investigate review all changes made to the code and revise something that might of effected that dip.

Non Product Reports: Data, Referrals & Status

Data: If you’re not a technical person then it’s probably best to provide a simple reporting interface on top of the data that business type folks can run and export to csv. This is typically used to send email updates about the product, contact cohorts of users or giving notice to system outages.

Referrals: We have this since we are currently in closed beta and want to slowly refine our on-boarding process and metrics. It’s just a simple way to throttle users into our app and refine things in a structured way. You may not require this.

Status: If you have any integration with 3rd party API’s or host your app out on the cloud, then you’ll want to instrument some system monitoring for response times, status (up/down) and other types of performance. This is the first place we look before reporting a bug from a user – if one of our 3rd party API are misbehaving then there’s a good chance it’s related.

Right Time, Right Action

Having an amazing admin area is something you should build over time. Initially it’s best to spend time on product reports and a few non-product screens. We usually add things as we feel the need to streamline a process or learn more about a specific use case. It all depends on the product metrics and goals, however, I do believe that having everything consolidated into one area makes (vs 7 diff apps you log into) makes the world of difference.

How about you, what are some of the data points you monitor or report on in your admin dashboard? Love to see some screenshots or ideas below in the comments.