Pusher Logo

    Building a realtime analytics dashboard for a Laravel application

    To follow this tutorial you will need PHP 7.2 or higher, with the MongoDB driver enabled. You will also need MongoDB and Composer installed, and a Pusher account.

    In today’s world, it’s important for website administrators and webmasters to have useful data regarding issues such as the performance of their sites. This helps them to be proactive in tackling issues with their sites. In this tutorial, we’ll build a middleware that logs all requests made to our application and pushes updated analytics on those in realtime to a dashboard. Here’s a preview of our app in action:

    laravel-analytics-demo

    Prerequisites

    1. PHP 7.2 or higher, with the MongoDB driver installed. You can find installation instructions here.
    2. Composer
    3. MongoDB (version 3.4 or higher). Get it here.
    4. A Pusher account. Create one here.

    Setting up the app

    Laravel by default uses SQL databases as the backend for its Eloquent models, but we’re using MongoDB in this project, so we’ll start off with a Laravel installation configured to use MongoDB. Clone the repo by running:

        git clone https://github.com/shalvah/laravel-mongodb-starter.git

    You can also download the source directly from this link.

    Then cd into the project folder and install dependencies:

        composer install

    Lastly, copy the .env.example to a new file called .env. Run the following command to generate an application encryption key:

        php artisan key:generate

    Note: If your MongoDB server requires a username and password, add those in your .env file as the DB_USERNAME and DB_PASSWORD respectively.

    Logging all requests

    We’ll create a middleware that logs every request to our database. Our middleware will be an "after” middleware, which means it will run after the request has been processed but just before sending the response. We’ll store the following details:

    • The relative URL (for instance, /users)
    • The HTTP method (for instance, “GET”)
    • The time it took to respond to the request
    • The day of the week
    • The hour of day,

    Let’s create our RequestLog model. Create the file app/Models/RequestLog.php with the following content:

        <?php
    
    namespace App\Models;
    
    use Jenssegers\Mongodb\Eloquent\Model;
    
    class RequestLog extends Model
    {
        protected $guarded = [];
    }

    Then create the file app/Http/Middleware/RequestLogger.php with the following content:

        <?php
    
    namespace App\Http\Middleware;
    
    use App\Models\RequestLog;
    use Carbon\Carbon;
    use Closure;
    
    class RequestLogger
    {
        public function handle(\Illuminate\Http\Request $request, Closure $next)
        {
            $response = $next($request);
    
            if ($request->routeIs('analytics.dashboard')) {
                return $response;
            }
    
            $requestTime = Carbon::createFromTimestamp($_SERVER['REQUEST_TIME']);
            $request = RequestLog::create([
                'url' => $request->getPathInfo(),
                'method' => $request->method(),
                'response_time' => time() - $requestTime->timestamp,
                'day' => date('l', $requestTime->timestamp),
                'hour' => $requestTime->hour,
            ]);
    
            return $response;
        }
    }

    The if condition above prevents us from logging anything if the request is to view the analytics dashboard.

    Now, let’s attach the middleware to all our routes. In your app/Http/Kernel.php, add the middleware class to the $middleware array:

        protected $middleware = [
        // ...
        // ...
        \App\Http\Middleware\RequestLogger::class,
    ];

    We won’t be using any authentication in our app, but we need some of the frontend scaffolding Laravel provides, so we’ll run this command:

        php artisan make:auth

    Displaying our analytics

    Let’s add the route for the analytics dashboard. Add the following to the end of your routes/web.php:

        Route::get('/analytics', 'AnalyticsController@index')->name('analytics.dashboard');

    Next, we’ll create a class that retrieves our analytics using MongoDB aggregations. Create the file app/Services/AnalyticsService.php with the following content:

        <?php
    
    namespace App\Services;
    
    
    use App\Models\RequestLog;
    use Jenssegers\Mongodb\Collection;
    
    class AnalyticsService
    {
    
        public function getAnalytics()
        {
    
            $perRoute = RequestLog::raw(function (Collection $collection) {
                return $collection->aggregate([
                    [
                        '$group' => [
                            '_id' => ['url' => '$url', 'method' => '$method'],
                            'responseTime' => ['$avg' => '$response_time'],
                            'numberOfRequests' => ['$sum' => 1],
                        ]
                    ]
                ]);
            });
            $requestsPerDay = RequestLog::raw(function (Collection $collection) {
                return $collection->aggregate([
                    [
                        '$group' => [
                            '_id' => '$day',
                            'numberOfRequests' => ['$sum' => 1]
                        ]
                    ],
                    ['$sort' => ['numberOfRequests' => 1]]
                ]);
            });
            $requestsPerHour = RequestLog::raw(function (Collection $collection) {
                return $collection->aggregate([
                    [
                        '$group' => [
                            '_id' => '$hour',
                            'numberOfRequests' => ['$sum' => 1]
                        ]
                    ],
                    ['$sort' => ['numberOfRequests' => 1]]
                ]);
            });
            return [
                'averageResponseTime' => RequestLog::avg('response_time'),
                'statsPerRoute' => $perRoute,
                'busiestDays' => $requestsPerDay,
                'busiestHours' => $requestsPerHour,
                'totalRequests' => RequestLog::count(),
            ];
        }
    }

    Here are the analytics we’re gathering:

    • averageResponseTime ****is the average time taken by our routes to return a response.
    • statsPerRoute contains information specific to each route, such as the average response time and number of requests.
    • busiestDays contains a list of all the days, ordered by the number of requests per day.
    • busiestHours contains a list of all the hours, ordered by the number of requests per hour.
    • totalRequests is the total number of requests we’ve gotten.

    Then create the controller, app/Http/Controllers/AnalyticsController.php with the following content:

        <?php
    
    namespace App\Http\Controllers;
    
    use App\Services\AnalyticsService;
    
    class AnalyticsController extends Controller
    {
        public function index(AnalyticsService $analyticsService)
        {
            $analytics = $analyticsService->getAnalytics();
            return view('analytics', ['analytics' => $analytics]);
        }
    }

    Now let’s create the markup. Create the file resources/views/analytics.blade.php with the following content:

        @extends('layouts.app')
    
    @section('content')
        <div class="container" id="app">
            <div class="row">
                <div class="col-md-5">
                    <div class="card">
                    <div class="card-body">
                        <h5 class="card-title">Total requests</h5>
                        <div class="card-text">
                            <h3>@{{ totalRequests }}</h3>
                        </div>
                    </div>
                    </div>
                </div>
                <div class="col-md-5">
                    <div class="card">
                    <div class="card-body">
                        <h5 class="card-title">Average response time</h5>
                        <div class="card-text">
                            <h3>@{{ averageResponseTime.toFixed(4) }} seconds</h3>
                        </div>
                    </div>
                    </div>
                </div>
            </div>
    
            <div class="row">
                <div class="col-md-5">
                    <div class="card">
                    <div class="card-body">
                        <h5 class="card-title">Busiest days of the week</h5>
                        <div class="card-text" style="width: 18rem;" v-for="day in busiestDays">
                            <ul class="list-group list-group-flush">
                                <li class="list-group-item">
                                    @{{ day._id }} (@{{ day.numberOfRequests }} requests)
                                </li>
                            </ul>
                        </div>
                    </div>
                </div>
                </div>
                <div class="col-md-5">
                    <div class="card">
                    <div class="card-body">
                        <h5 class="card-title">Busiest hours of day</h5>
                        <div class="card-text" style="width: 18rem;" v-for="hour in busiestHours">
                            <ul class="list-group list-group-flush">
                                <li class="list-group-item">
                                    @{{ hour._id }} (@{{ hour.numberOfRequests }} requests)
                                </li>
                            </ul>
                        </div>
                    </div>
                    </div>
                </div>
            </div>
    
            <div class="row">
                <div class="col-md-5">
                    <div class="card">
                    <div class="card-body">
                        <h5 class="card-title">Most visited routes</h5>
                        <div class="card-text" style="width: 18rem;" v-for="route in statsPerRoute">
                            <ul class="list-group list-group-flush">
                                <li class="list-group-item">
                                    @{{ route._id.method }} @{{ route._id.url }} (@{{ route.numberOfRequests }} requests)
                                </li>
                            </ul>
                        </div>
                    </div>
                    </div>
                </div>
                <div class="col-md-5">
                    <div class="card">
                    <div class="card-body">
                        <h5 class="card-title">Slowest routes</h5>
                        <div class="card-text" style="width: 18rem;" v-for="route in statsPerRoute">
                            <ul class="list-group list-group-flush">
                                @{{ route._id.method }} @{{ route._id.url }} (@{{ route.responseTime || 0}} s)
                            </ul>
                        </div>
                    </div>
                    </div>
                </div>
            </div>
        </div>
    @endsection
    
    <script>
        window.analytics = @json($analytics);
    </script>

    We’ll be using Vue.js to automatically bind data on the view, so the @{{…}} around expressions above signify to Laravel’s templating engine that these are JavaScript, not PHP, expressions. Let’s write the Vue code. Replace the code in your resources/assets/js/app.js with the following:

        require('./bootstrap');
    
    window.Vue = require('vue');
    const app = new Vue({
        el: '#app',
    
        data: window.analytics
    });

    Updating the dashboard in realtime

    Now let’s update our request logging middleware so that it broadcasts the updated analytics to the frontend whenever a new request is made. We’ll be making use of Laravel’s event broadcasting for this.

    First, we’ll create an AnalyticsUpdated event. Create the file app/Events/AnalyticsUpdated.php with the following content:

        <?php
    
    namespace App\Events;
    
    use App\Services\AnalyticsService;
    use Illuminate\Broadcasting\Channel;
    use Illuminate\Contracts\Broadcasting\ShouldBroadcastNow;
    use Illuminate\Queue\SerializesModels;
    use Illuminate\Foundation\Events\Dispatchable;
    use Illuminate\Broadcasting\InteractsWithSockets;
    
    class AnalyticsUpdated implements ShouldBroadcastNow
    {
        use Dispatchable, InteractsWithSockets, SerializesModels;
    
        // The updated analytics
        public $analytics;
    
        public function __construct()
        {
            $this->analytics = (new AnalyticsService())->getAnalytics();
        }
        public function broadcastOn()
        {
            return new Channel('analytics');
        }
    }

    Now, update your app/Http/Middleware/RequestLogger.php so it looks like this:

        <?php
    
    namespace App\Http\Middleware;
    
    use App\Events\AnalyticsUpdated;
    use App\Models\RequestLog;
    use Carbon\Carbon;
    use Closure;
    
    class RequestLogger
    {
        public function handle(\Illuminate\Http\Request $request, Closure $next)
        {
            $response = $next($request);
    
            if ($request->routeIs('analytics.dashboard')) {
                return $response;
            }
    
            $requestTime = Carbon::createFromTimestamp($_SERVER['REQUEST_TIME']);
            $request = RequestLog::create([
                'url' => $request->getPathInfo(),
                'method' => $request->method(),
                'response_time' => time() - $requestTime->timestamp,
                'day' => date('l', $requestTime->timestamp),
                'hour' => $requestTime->hour,
            ]);
    
            // we broadcast the event
            broadcast(new AnalyticsUpdated());
            return $response;
        }
    }

    We need to do a few things to configure event broadcasting in your app. First, to enable event broadcasting, open up your config/app.php and uncomment the line in the providers array that contains the BroadcastServiceProvider:

        'providers' => [
        ...
        // uncomment the line below
        // App\Providers\BroadcastServiceProvider::class,
        ...
    ],

    Then sign in to your Pusher dashboard and create a new app. Copy your app credentials from the App Keys section and add them to your .env file:

        PUSHER_APP_ID=your-app-id
    PUSHER_APP_KEY=your-app-key
    PUSHER_APP_SECRET=your-app-secret
    PUSHER_APP_CLUSTER=your-app-cluster

    Then change the value of BROADCAST_DRIVER in your .env file to pusher:

        BROADCAST_DRIVER=pusher

    Open up the file config/broadcasting.php. Within the pusher key of the the connections array, set the value of the encrypted option to false:

        return [
        // ...
        'connections' => [
            'pusher' => [
                'driver' => 'pusher',
                'key' => env('PUSHER_APP_KEY'),
                'secret' => env('PUSHER_APP_SECRET'),
                'app_id' => env('PUSHER_APP_ID'),
                'options' => [
                    'cluster' => env('PUSHER_APP_CLUSTER'),
                    'encrypted' => false, 
                 ],
             ],
        ]
    ];

    Note: Laravel sometimes caches old configuration, so for the project to see your new configuration values, you might need to run the command php artisan config:clear

    Then install the Pusher PHP library by running:

        composer require pusher/pusher-php-server "~3.0"

    On the frontend, we need to install Laravel Echo and the Pusher JavaScript library. Do this by running:

        # install existing dependencies first
    npm install
    npm install pusher-js laravel-echo --save

    Next, uncomment the following lines in your resources/assets/js/bootstrap.js:

        // import Echo from 'laravel-echo'
    // 
    // window.Pusher = require('pusher-js');
    //
    // window.Echo = new Echo({
    //     broadcaster: 'pusher',
    //     key: process.env.MIX_PUSHER_APP_KEY,
    //     cluster: process.env.MIX_PUSHER_APP_CLUSTER,
    //     encrypted: true
    // });

    Now open up your resources/assets/js/app.js and add the following code to the end:

        Echo.channel('analytics')
        .listen('AnalyticsUpdated', (event) => {
            Object.keys(event.analytics).forEach(stat => {
                window.analytics[stat] = event.analytics[stat];
            })
        });

    Here we listen for the AnalyticsUpdated and update each statistic accordingly. Since we’ve bound the data item of the Vue instance to window.analytics earlier, by changing a value in window.analytics, we can be sure that Vue will automatically re-render with the updated values.

    Now run npm run dev to compile and build our assets.

    For us to test our app, we need some routes to visit. These routes should take different amounts of time to load, so we can see the effect on our statistics. Let’s add a dummy route that waits for how many seconds we tell it to. Visiting /wait/3 will wait for three seconds, /wait/1 for one second and so on. Add this to the end of your routes/web.php:

        Route::get('/wait/{seconds}', function ($seconds) {
        sleep($seconds);
        return "Here ya go! Waited for $seconds seconds";
    });

    Let’s see the app in action. Start your MongoDB server by running mongod. (On Linux/macOS, you might need to run it as sudo).

    Then start your app by running:

        php artisan serve

    Visit your analytics dashboard at http://localhost:8000/analytics. Then play around with the app by visiting a few pages (the wait URL with different values for the number of seconds) and watch the stats displayed on the dashboard change in realtime.

    Conclusion

    In this article, we’ve built a middleware that tracks every request, a service that computes analytics for us based on these tracks, and a dashboard that displays them. Thanks to Pusher, we’ve been able to make the dashboard update in realtime as requests come in. The full source code is available on GitHub.

    Laravel PHP

    1 May 2018

    by Shalvah Adebayo