Sending Azure Eventhub Logs to Scalyr

ATTENTION: This solution has been deprecated

Please see our current Azure integration here: https://support.scalyr.com/hc/en-us/articles/4408683030291-Azure-Diagnostic-Logs-in-Scalyr

Scalyr supports integration with AWS CloudWatch for a while now. As we continue to invest in the cloud, it is extremely important to us that Scalyr offers the most convenient solutions to get your cloud application and infrastructure logs into our platform, so you can simply monitor everything in one single place.

I had previously written the steps for sending Google Cloud Stackdriver logs to Scalyr. Recently, I applied the same approach to Microsoft Azure and it is working pretty well for me. I simply configure Eventhub to receive Azure activity logs and then bind an Azure function to send those logs to Scalyr. Here are the steps…

Prerequisite: You already have an Azure Eventhub created and it is receiving logs from at least one Azure service.

Go to the Function App to Create a Function

Selecting “Basics” on the Function App page and fill in the following parameters

• Subscription: <CHOSE YOUR OWN SUBSCRIPTION>
• Resource Group: <CHOOSE THE RESOURCE GROUP TO HOST THE FUNCTION APP>
• Function App name: <DEFINE YOUR OWN NAME>
• Publish: Code
• Runtime Stack: Node.js
• Region: <CHOOSE THE SAME REGION WHERE THE EVENTHUB IS IN>

Setup Azure Eventhub Trigger

Select the function app you just created and click + from the “Functions” dropdown menu.

Select “Azure Event Hub Trigger” and fill in the following parameters to connect the function with the eventhub. Eventhub extension is required to be installed if the trigger has never been used. You can just follow the pop-up window’s instructions to install it.

• Name: <DEFINE YOUR OWN FUNCTION NAME>
• Event Hub connection: <eventhub_name>_RootManageSharedAccessKey_EVENTHUB>
• Event Hub consumer group: $Default
• Event Hub name: <YOUR EVENTHUB NAME>
•  

Create Function app Script

Add the following script to index.js. You should replace variables “token”, “logfile” and “host” based on your own environment’s configurations.

const querystring = require('querystring');
var https = require('https');
const util = require('util')
module.exports =  function (context, eventHubMessages) {
  context.log(`JavaScript eventhub trigger function called for message array ${eventHubMessages}`);
  //convert eventhub message to a string
  var payload = JSON.stringify({
    eventHubMessages
  })
  var host = 'myhost'
  var logfile = 'mylogfile'
  var token = 'XXXX'
var options = {
  hostname: 'scalyr.com',
  port: 443,
  path: `/api/uploadLogs?token=${token}&host=${host}&logfile=${logfile}`,
  method: 'POST',
  body: payload,
  headers: {
       'Content-Type': 'application/json',
     }
};
var myReq = https.request(options, (res) => {
  context.log('statusCode:', res.statusCode);
  res.on('data', (d) => {context.log('data:', d);
    process.stdout.write(d);
  });
});
myReq.write(payload);
myReq.on('error', (e) => {
  console.error(e);
});
myReq.end();
};

Clicking “Save” and “Run” sends a raw log message “Test Message” to Scalyr. Any Aure services that push logs to eventhub should trigger this Function app and forward the logs to Scalyr.

Although this is just a prototype, you can still follow the instruction to set it up yourself and get some of your Azure logs to Scalyr. We’re working on the full version so stay tuned and feel free to reach out if you are interested in hearing more about it.