Installation of the SplunkForSquid app for Splunk

Applies to Platform: UTM 2.5, UTM 3.0.X
Updated on: 2nd of November 2012

This lesson shows how to configure Splunk on your Endian UTM Appliance to analyse the log file of squid (HTTP proxy) and generate and show reports about the browsing activities of each user.

Requirements

To start the analysis of squid log files on an Endian UTM Appliance you need:

• SplunkforSquid app
• Splunk configured and running on your Endian UTM appliance on port 8000 (if you changed the port on which Splunk listens, use that value instead of 8000)
• HTTP Proxy running, with logging enabled on your Endian UTM appliance

Preliminaries

You need to download the SplunkForSquid.tar.gz file for the splunk website and save it on your local workstation. You will need to upload it later.

Before starting the configuration of the squid app for splunk, you should activate HTTP proxy logging. If you do not have yet done it, log in to the web interface of your Endian UTM Appliance at http://GREENIP:10443/manage/dashboard and activate logging for the HTTP proxy under Menubar > Proxy > Log settings, ticking the HTTP proxy logging option. The more detailed you want the splunk reports to be, the more logging option in the box you should activate.

Installation of the Squid App

Log in to the splunk web interface, located on the Endian UTM Appliance at the location http://GREENIP:8000/.

Once you've logged in, you should install the squid app: Go to Splunk home > apps > install app from file > browse, choose the SplunkForSquid.tar.gz, and click on Upload.

Customisation of the Squid App

For a better experience, you should now edit and customise the app, starting with the default views. Go to Splunk home > Other Configurations > User interface > Views > App context > Splunk for Squid (SplunkforSquid) > request_search. You will see an XML file that you should backup (so in case anything goes wrong you can use that one) and replace with the following:

<form>
  <label>Requests search</label>

  <searchTemplate>sourcetype="squid" username="$username$" clientip="$clientip$" uri_host="$uri_host$"</searchTemplate>

  <fieldset>
    <input type="text" token="username">
      <label>Username</label>
      <seed>*</seed>
    </input>

    <input type="text" token="clientip">
      <label>Client IP</label>
      <seed>*</seed>
    </input>

    <input type="text" token="uri_host">
      <label>Destination hostname</label>
      <seed>*</seed>
    </input>

    <input type="text" token="uri_path">
      <label>Requested URL</label>
      <seed>*</seed>
    </input>

    <input type="time"/>
  </fieldset>

  <row>
    <chart>
      <title>Requests over time for result set</title>
      <searchPostProcess>timechart count as "Requests"</searchPostProcess>
      <option name="charting.chart">column</option>
    </chart>
  </row>

  <row>
    <chart>
      <title>Top client IPs in result set</title>
      <searchPostProcess>top 10 clientip</searchPostProcess>
      <option name="charting.chart">pie</option>
    </chart>
    <chart>
      <title>Top destination sites in result set</title>
      <searchPostProcess>top 10 uri_host</searchPostProcess>
      <option name="charting.chart">pie</option>
    </chart>
     <chart>
      <title>Top user in result set</title>
      <searchPostProcess>top 10 username</searchPostProcess>
      <option name="charting.chart">pie</option>
    </chart>

  </row>

  <row>
    <table>
      <title>Requests in result set</title>
      <!-- searchPostProcess>sort - _time | fields _time, clientip, uri_host, uri_path</searchPostProcess -->
      <fields>_time, clientip, uri_host, uri_path, bytes</fields>
      <option name="showPager">true</option>
      <option name="count">30</option>
      <option name="displayRowNumbers">false</option>
      </table>
  </row>
</form>

Click on Save to store the new view. You can now create custom graphs to display reports on the user's browsing activities. Go to Splunk Home > Other Configurations > User interface > Views > App context > Splunk for Squid (SplunkforSquid) > New and insert the following three new view names:

• Insert userin View name.
• Insert the following XML code in View xml, then press Save.

  <?xml version='1.0' encoding='utf-8'?>
  <dashboard>
    <label>user</label>
  </dashboard>

• Insert top users in View name.
• Insert the following XML code in View xml, then press Save.

  <?xml version='1.0' encoding='utf-8'?>
  <dashboard>
    <label>top users</label>
  </dashboard>

• Insert top userin View name.
• Insert the following XML code in View xml, then press Save.

  <?xml version='1.0' encoding='utf-8'?>
  
  <dashboard>
    <label>top user</label>
  </dashboard>

Configuration of the data sources

Data sources to feed the Squid app can be configured under Splunk Home > Other Configurations > Data inputs > Files & directories > New.
Here, a wizard will help you in the definition of the new data sources.

• In preview Data, select option Skip preview, then click on Continue.
• To activate the monitoring of log files, in Add New select Continuously index data from a file or directory this Splunk instance can access
• In full path of your data provide the file name containing the log messages: /var/log/squid/access.log_short
• Finally, tick more settings, then from the set source type drop-down menu choose the option manual, and in Source Type enter squid.
• Click on Save to store the configuration.

You are now done. Enter in your browser the address http://GREENIP:8000/en-US/app/SplunkforSquid/dashboard. Here you can select a time interval and see the corresponding graphs.

/