Improving Observability With Cloudsmith Logs

Observability is emerging as critical DevOps practice to predict, prevent, identify, and address issues in the distributed, dynamic, complex, and constantly evolving applications and infrastructures that characterize modern software systems. Wisely, many teams are adopting observability tools like Datadog, Elastic, and AWS Athena to aggregate logs from their build tooling and better visualize and understand the state of their entire system.

But, observability tools are only as good as the logs they consume. 

A high-caliber artifact management solution should be a rich, central source of information about your software supply chain; after all, it hosts your software and your dependencies. This is why we invest heavily in the quality of Cloudsmith’s logs and make sure they’re easy for you to extract your logs and integrate into your observability tools.

Sharpening your vision with Cloudsmith logs 

Our logs give you granular visibility into your entire lifecycle of artifacts, user interactions, and system behavior so you can proactively monitor, debug, and optimize your software supply chain and operations. We provide these three main types of logs:

Reduce risk and react faster with audit logs

Our organizational and repository audit logs are essential for robust cybersecurity and compliance strategies. Organizational audit logs capture events across your organization, including the creation and deletion of repositories and modifications to settings.

Repository audit logs offer a timeline of non-package actions within a repo (typically administrative activities), like modifying retention rules or creating an entitlement token.

Together, they offer a comprehensive view, and analyzing these detailed records can be extremely useful for the following:

Security Incident Management

• Identify and respond to unauthorized access attempts or suspicious activities in the audit logs.
• Trigger alerts for any unusual patterns or anomalies in user activities that indicate a potential security threat.

Compliance Monitoring

• Track changes in repository settings or retention rules to ensure compliance with organizational policies.
• Monitor and audit user actions to meet compliance requirements and regulatory standards.

Troubleshooting

• Investigate and troubleshoot specific issues reported by users or identified through audit logs.
• Utilize historical audit logs to trace changes and events leading up to an issue and rectify it quickly.
• Identify and resolve issues promptly by reviewing audit logs for any operational discrepancies or errors.

Resource Planning

• Use audit logs to track repository creation/deletion and help in capacity planning.

Improve OpEx and product planning with client logs

Our client logs help you understand how your builds are being consumed by providing a record of every package downloaded by your users, along with information such as date and time, user agent, and IP address.

You can leverage these logs to improve product and operational decisions, for example:

Operational Efficiency

• Analyze client logs to understand how packages are being consumed to help you optimize infrastructure and resource allocation.
• Monitor package downloads and associated bandwidth usage.
• Implement alerts based on download thresholds to proactively manage and optimize cloud costs.
• Analyze download trends to align resource provisioning with actual usage, preventing unnecessary expenditure and optimizing your cloud bill.

Product Development Insights

• Use client logs to identify popular packages, enabling prioritization in product development efforts.
• Analyze download trends by country and usage by package versions to inform feature improvements.

Anticipate capacity changes with package logs

Our package logs provide a detailed record of all package-related events within a repository.

These include unique event IDs, package IDs, event types, dates/times, user IP addresses, and user account details, and they’re useful for:

Resource Planning

• Analyze package logs to understand resource utilization trends and plan infrastructure scaling accordingly.
• Analyze user behavior for:
  • Usage Patterns: Identify patterns in package downloads, helping you understand peak usage times and plan for increased capacity during high-demand periods.
  • Geographical Trends: Analyze user locations from IP addresses, allowing you to anticipate regional variations in demand and optimize resources accordingly.
  • Popular Packages: Identify frequently downloaded packages, enabling proactive scaling for high-demand software.

Compatible for convenient extraction and analysis

You can view all of our logs in your Cloudsmith UI, and we provide help documentation for our organizational audit logs, repository audit logs, client logs and package logs. But you can also easily export and integrate them with your observability tools. 

Extract

With Cloudsmith you can:

• export to an AWS S3 bucket; and 
• access real-time audit logs via the Cloudsmith API for your Organization and your Repositories. 

Analyze

Most observability tools can consume logs from an AWS S3 bucket, either by ingesting data from custom RESTful API or through integrations. 

Datadog

Datadog is a cloud-based monitoring and analytics platform that can ingest and analyze logs, providing insights into system performance. The Datadog-Cloudsmith integration, ingests audit logs information, incorporates other status details from Cloudsmith, and displays them on a Datadog dashboard.

Elasticsearch 

Part of the Elastic Stack, Elasticsearch is a distributed search and analytics engine that can efficiently analyze and visualize logs consumed from an S3 bucket. Alternatively, Elastic agent can be used for collecting custom HTTPJSON events

Splunk

Splunk is a widely-used platform for searching, monitoring, and analyzing machine-generated data, including logs. You can use Splunk to ingest Cloudsmith logs from the AWS S3 bucket. 

Amazon Athena

A serverless query service, Amazon Athena allows you to analyze data directly in an Amazon S3 bucket using standard SQL. We have documentation on how to use AWS Athena to extract insights from our Client logs.

Cloudsmith logs are crucial for enhancing security, compliance, and operational efficiency in your software supply chain. Explore the power of Cloudsmith today to empower your organization with actionable insights. Or contact us with any questions. We’re more than happy to help.

Liked this article? Don\'t be selfish (:-), share with others:   Tweet