Audit Trail Streaming
As you navigate the intricacies of managing your application's data, you might realize the significance of effective audit trail management. This article delves into how Descope facilitates seamless streaming of audit logs to third-party services like Amazon S3 and Datadog. By harnessing the power of Descope's connectors, you can effortlessly integrate and relay your audit records to these platforms, optimizing storage, analytics, and security.
Why Stream the Audit Logs?
In today's digital landscape, the integrity and security of data play pivotal roles in ensuring the trustworthiness of systems and the confidence of users. Audit logs are critical to this ecosystem, capturing a detailed record of an application's activities, transactions, and interactions. While maintaining these logs within the application is crucial, streaming them to third-party services like Amazon S3 or Datadog amplifies their value.
By offloading logs to such services, organizations can benefit from enhanced storage scalability, improved resilience against data loss, and the ability to leverage advanced analytics and monitoring tools specific to these platforms. Moreover, storing logs on a third-party service can be a lifesaver in system breaches, ensuring that vital forensic data remains intact and uncompromised.
Audit Streaming with Connectors
Descope enables you to stream your audit trail to a third party by implementing connectors. Within Descope, you can
configure connectors to various third-party services to stream the audit trail. You can find the supported connectors
by searching audit on the connectors page.
This knowledge base article will show an example of configuring the Amazon S3 connector; however, the steps are similar to the other third-party services.
Filtering Audit Stream
Descope allows you to filter the audit stream based on tenants as well as actions. This is helpful if you want to send the audit stream from your tenants to different endpoints for your customers, or if you have certain actions you wish to stream, or not to stream.
Amazon S3 Example
Configure the Connector
To configure with Amazon S3, navigate to the connectors page and select the
Amazon S3 connector. You will be guided through configuring the connector on this page. The Amazon S3 connector
requires an access key ID and access key secret whereas other audit streaming connectors may require different
credentials such as API Keys.
Specifically to Amazon, you can find how to create access keys within Amazon's Documentation.
Once you have gathered the necessary credentials, you are now ready to configure the connector. The Help Guide
on the right hand side has details about the necessary items and configurations as well. A sample configuration
can be seen below. This example also shows the successful test of the connector.
Viewing Audit Objects in Amazon S3
With the Amazon S3 use case, the audit trail is saved as json objects. Other connectors like Datadog will store them in a readable format as you would expect from other third-parties. With Amazon S3, you will need to open the objects to view the contents.
The objects are stored within a directory based on the project ID, and then further sorted to directories based on the data. Below you can see an example format of how the objects would be stored within your S3 bucket.
![An example of the date formatted directory structure when streaming Descope audit logs to Amazon S3](/assets/example-amazon-s3-directory-structure-2.webp