Introduction
The data available in the ProsperOps Console enables FinOps practitioners to understand the savings outcomes realized by our automation platform. In certain cases, customers may want to programmatically integrate with ProsperOps data and/or perform ad hoc analysis, e.g. incorporate Effective Savings Rate (ESR) into internal FinOps dashboards, integrate Intelligent Showback data into showback/chargeback pipelines, compare historical monthly savings performance across multiple AWS organizations, etc. Data Export enables these use cases.
Data Export is a free, optional feature for AWS organizations managed by ProsperOps Autonomous Discount Management. Once enabled, it writes ProsperOps data (in CSV format -- see here for details) to an S3 bucket daily. Exports include all historical data in the ProsperOps platform (e.g. if you have been a ProsperOps customer for 3 years and enable Data Export, your export files will include 3 years of data). You can think of and interact with Data Export files much like you do AWS Cost and Usage Report (CUR) files. No ProsperOps-specific APIs or API keys are required. You can leverage native AWS permissioning and services to integrate. For example, each time the ProsperOps Data Export files are updated, you might leverage S3 triggers to automatically invoke a Lambda function that reads in the new data and takes action on it. In short, AWS customers can use existing models and tooling they are already familiar with to integrate ProsperOps data.
The following data sets are currently available as part of Data Export. As new data sets are made available for export, they will be added below.
- Autonomous Discount Management for AWS Compute
- Savings Dashboard Monthly
- Intelligent Showback
Configuration
Exports are made available at the AWS organization level. If you have more than a single AWS organization managed by Autonomous Discount Management, exports must be configured for each organization. To configure Data Export for a given organization, follow these steps:
1. Open the ProsperOps Console, select the AWS Organization where you'd like to enable Data Export, and click on Settings under CONFIGURATION in the left navigation.
2. Click Configure in the Data Export settings panel and enter the S3 bucket name (must be globally unique per S3 naming rules) that will house the export files. If the S3 bucket doesn't already exist, please create it now. Click Next to continue.
NOTE: For data security and privacy purposes, since ProsperOps exports include organization wide data, the S3 bucket must exist in the management account of the organization (any region is fine).
3. In order for ProsperOps to write exports to the configured S3 bucket, the ProsperOps role in the management account must be updated with the necessary permissions. The provided IAM policy includes these permissions. Click Copy and apply the updated IAM policy to the ProsperOps role as appropriate. Once complete, click Validate & Enable.
If the S3 bucket does not exist or permissions have not been applied correctly, you will receive a detailed error message similar to the one shown below. Should that happen, correct the issue and click Validate & Enable again. If the issue persists, please contact us for assistance.
4. Once access has been validated, Data Export will be enabled and exports will begin within 24 hours. Last Export will show the last date-time an export was written (times shown are local).
Managing Setting Changes
- To pause or turn off Data Export, click the Disable button.
- To re-enable Data Export, click the Configure button and follow the steps above to reconfigure. You may use the same S3 bucket and permissions that were previously configured. Once enabled, gap data is backfilled and you will have a complete historical data set.
- To change your S3 bucket name, you must turn off Data Export then re-enable using the new S3 bucket name. As part of the setup, you will need to update the ProsperOps role IAM policy and validate access.