AWS CUR 2.0 Manual Setup Guide

A step-by-step guide to manually create AWS Cost and Usage Report 2.0 with Athena integration.

Written By Dotan Cohen

Last updated About 2 months ago

A step-by-step guide to manually create AWS Cost and Usage Report 2.0 with Athena integration.

Prerequisites

  • AWS Account with billing access
  • Permissions: S3, Glue, Athena, BCM Data Exports
  • AWS CLI configured (optional, for CLI steps)

Step 1: Create S3 Bucket

Console:

  1. Go to S3 → Create bucket
  2. Bucket name: wiv-cur2-{account-id} (e.g., wiv-cur2-123456789012)
  3. Region: us-east-1 (recommended for CUR)
  4. Keep defaults → Create bucket

CLI:

aws s3 mb s3://wiv-cur2-123456789012 --region us-east-1

Step 2: Add S3 Bucket Policy

Console:

  1. Go to your bucket → Permissions → Bucket policy
  2. Add this policy (replace 123456789012 with your account ID):
{ "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": { "Service": [ "billingreports.amazonaws.com", "bcm-data-exports.amazonaws.com" ] }, "Action": [ "s3:PutObject", "s3:GetBucketPolicy" ], "Resource": [ "arn:aws:s3:::wiv-cur2-123456789012", "arn:aws:s3:::wiv-cur2-123456789012/*" ], "Condition": { "StringLike": { "aws:SourceArn": [ "arn:aws:cur:us-east-1:123456789012:definition/*", "arn:aws:bcm-data-exports:us-east-1:123456789012:export/*" ], "aws:SourceAccount": "123456789012" } } } ] }

CLI:

aws s3api put-bucket-policy --bucket wiv-cur2-123456789012 --policy file://bucket-policy.json

Step 3: Create CUR 2.0 Export

Console:

  1. Go to Billing and Cost Management → Data Exports
  2. Click Create export
  3. Select Standard data export (CUR 2.0)
  4. Configure: Export name: Wiv-CUR2; Select all columns or use this query:
    SELECT * FROM COST_AND_USAGE_REPORT
  5. Table configurations: TIME_GRANULARITY: HOURLY; INCLUDE_RESOURCES: TRUE; INCLUDE_SPLIT_COST_ALLOCATION_DATA: TRUE
  6. Destination: S3 bucket: wiv-cur2-123456789012; S3 prefix: wiv
  7. Click Create export

CLI:

aws bcm-data-exports create-export \ --export '{ "Name": "Wiv-CUR2", "DataQuery": { "QueryStatement": "SELECT bill_bill_type, bill_billing_entity, bill_billing_period_end_date, bill_billing_period_start_date, bill_invoice_id, bill_invoicing_entity, bill_payer_account_id, bill_payer_account_name, capacity_reservation_capacity_reservation_arn, capacity_reservation_capacity_reservation_status, capacity_reservation_capacity_reservation_type, cost_category, discount, discount_bundled_discount, discount_total_discount, identity_line_item_id, identity_time_interval, line_item_availability_zone, line_item_blended_cost, line_item_blended_rate, line_item_currency_code, line_item_legal_entity, line_item_line_item_description, line_item_line_item_type, line_item_net_unblended_cost, line_item_net_unblended_rate, line_item_normalization_factor, line_item_normalized_usage_amount, line_item_operation, line_item_product_code, line_item_resource_id, line_item_tax_type, line_item_unblended_cost, line_item_unblended_rate, line_item_usage_account_id, line_item_usage_account_name, line_item_usage_amount, line_item_usage_end_date, line_item_usage_start_date, line_item_usage_type, line_item_user_identifier, pricing_currency, pricing_lease_contract_length, pricing_offering_class, pricing_public_on_demand_cost, pricing_public_on_demand_rate, pricing_purchase_option, pricing_rate_code, pricing_rate_id, pricing_term, pricing_unit, product, product_comment, product_fee_code, product_fee_description, product_from_location, product_from_location_type, product_from_region_code, product_instance_family, product_instance_type, product_instancesku, product_location, product_location_type, product_operation, product_pricing_unit, product_product_family, product_region_code, product_servicecode, product_sku, product_to_location, product_to_location_type, product_to_region_code, product_usagetype, reservation_amortized_upfront_cost_for_usage, reservation_amortized_upfront_fee_for_billing_period, reservation_availability_zone, reservation_effective_cost, reservation_end_time, reservation_modification_status, reservation_net_amortized_upfront_cost_for_usage, reservation_net_amortized_upfront_fee_for_billing_period, reservation_net_effective_cost, reservation_net_recurring_fee_for_usage, reservation_net_unused_amortized_upfront_fee_for_billing_period, reservation_net_unused_recurring_fee, reservation_net_upfront_value, reservation_normalized_units_per_reservation, reservation_number_of_reservations, reservation_recurring_fee_for_usage, reservation_reservation_a_r_n, reservation_start_time, reservation_subscription_id, reservation_total_reserved_normalized_units, reservation_total_reserved_units, reservation_units_per_reservation, reservation_unused_amortized_upfront_fee_for_billing_period, reservation_unused_normalized_unit_quantity, reservation_unused_quantity, reservation_unused_recurring_fee, reservation_upfront_value, resource_tags, savings_plan_amortized_upfront_commitment_for_billing_period, savings_plan_end_time, savings_plan_instance_type_family, savings_plan_net_amortized_upfront_commitment_for_billing_period, savings_plan_net_recurring_commitment_for_billing_period, savings_plan_net_savings_plan_effective_cost, savings_plan_offering_type, savings_plan_payment_option, savings_plan_purchase_term, savings_plan_recurring_commitment_for_billing_period, savings_plan_region, savings_plan_savings_plan_a_r_n, savings_plan_savings_plan_effective_cost, savings_plan_savings_plan_rate, savings_plan_start_time, savings_plan_total_commitment_to_date, savings_plan_used_commitment, split_line_item_actual_usage, split_line_item_net_split_cost, split_line_item_net_unused_cost, split_line_item_parent_resource_id, split_line_item_public_on_demand_split_cost, split_line_item_public_on_demand_unused_cost, split_line_item_reserved_usage, split_line_item_split_cost, split_line_item_split_usage, split_line_item_split_usage_ratio, split_line_item_unused_cost, tags FROM COST_AND_USAGE_REPORT", "TableConfigurations": { "COST_AND_USAGE_REPORT": { "TIME_GRANULARITY": "HOURLY", "INCLUDE_RESOURCES": "TRUE", "INCLUDE_SPLIT_COST_ALLOCATION_DATA": "TRUE" } }, "DestinationConfigurations": { "S3Destination": { "S3Bucket": "wiv-cur2-123456789012", "S3Prefix": "wiv", "S3Region": "us-east-1", "S3OutputConfigurations": { "OutputType": "CUSTOM", "Format": "PARQUET", "Compression": "SNAPPY", "Overwrite": "OVERWRITE_REPORT" } } }, "RefreshCadence": { "Frequency": "SYNCHRONOUS" } }' \ --region us-east-1

Step 4: Wait for Data Delivery

  • First data delivery takes up to 24 hours
  • Data will appear at: s3://wiv-cur2-{account}/wiv/Wiv-CUR/data/BILLING_PERIOD=YYYY-MM/
  • Check status in Billing → Data Exports

Step 5: Create Glue Database (for existing customer skip to 7)

Console:

  1. Go to AWS Glue → Databases → Add database
  2. Database name: wivdb
  3. Click Create database

CLI:

aws glue create-database \ --database-input '{"Name": "wivdb"}' \ --region us-east-1

Step 6: Create Athena Workgroup

Console:

  1. Go to Athena → Workgroups → Create workgroup
  2. Name: WivWorkspace
  3. Query result location: s3://wiv-cur2-123456789012/query-results/
  4. Click Create workgroup

CLI:

aws athena create-work-group \ --name WivWorkspace \ --configuration "ResultConfiguration={OutputLocation=s3://wiv-cur2-123456789012/query-results/}" \ --region us-east-1

Step 7: Create Athena Table with Partition Projection

Run this in Athena (replace 123456789012 with your account ID):

CREATE EXTERNAL TABLE wivdb.wiv_cur2 ( bill_bill_type string, bill_billing_entity string, bill_billing_period_end_date timestamp, bill_billing_period_start_date timestamp, bill_invoice_id string, bill_invoicing_entity string, bill_payer_account_id string, bill_payer_account_name string, capacity_reservation_capacity_reservation_arn string, capacity_reservation_capacity_reservation_status string, capacity_reservation_capacity_reservation_type string, cost_category map<string,string>, discount map<string,double>, discount_bundled_discount double, discount_total_discount double, identity_line_item_id string, identity_time_interval string, line_item_availability_zone string, line_item_blended_cost double, line_item_blended_rate string, line_item_currency_code string, line_item_legal_entity string, line_item_line_item_description string, line_item_line_item_type string, line_item_net_unblended_cost double, line_item_net_unblended_rate string, line_item_normalization_factor double, line_item_normalized_usage_amount double, line_item_operation string, line_item_product_code string, line_item_resource_id string, line_item_tax_type string, line_item_unblended_cost double, line_item_unblended_rate string, line_item_usage_account_id string, line_item_usage_account_name string, line_item_usage_amount double, line_item_usage_end_date timestamp, line_item_usage_start_date timestamp, line_item_usage_type string, line_item_user_identifier string, pricing_currency string, pricing_lease_contract_length string, pricing_offering_class string, pricing_public_on_demand_cost double, pricing_public_on_demand_rate string, pricing_purchase_option string, pricing_rate_code string, pricing_rate_id string, pricing_term string, pricing_unit string, product map<string,string>, product_comment string, product_fee_code string, product_fee_description string, product_from_location string, product_from_location_type string, product_from_region_code string, product_instance_family string, product_instance_type string, product_instancesku string, product_location string, product_location_type string, product_operation string, product_pricing_unit string, product_product_family string, product_region_code string, product_servicecode string, product_sku string, product_to_location string, product_to_location_type string, product_to_region_code string, product_usagetype string, reservation_amortized_upfront_cost_for_usage double, reservation_amortized_upfront_fee_for_billing_period double, reservation_availability_zone string, reservation_effective_cost double, reservation_end_time string, reservation_modification_status string, reservation_net_amortized_upfront_cost_for_usage double, reservation_net_amortized_upfront_fee_for_billing_period double, reservation_net_effective_cost double, reservation_net_recurring_fee_for_usage double, reservation_net_unused_amortized_upfront_fee_for_billing_period double, reservation_net_unused_recurring_fee double, reservation_net_upfront_value double, reservation_normalized_units_per_reservation string, reservation_number_of_reservations string, reservation_recurring_fee_for_usage double, reservation_reservation_a_r_n string, reservation_start_time string, reservation_subscription_id string, reservation_total_reserved_normalized_units string, reservation_total_reserved_units string, reservation_units_per_reservation string, reservation_unused_amortized_upfront_fee_for_billing_period double, reservation_unused_normalized_unit_quantity double, reservation_unused_quantity double, reservation_unused_recurring_fee double, reservation_upfront_value double, resource_tags map<string,string>, savings_plan_amortized_upfront_commitment_for_billing_period double, savings_plan_end_time string, savings_plan_instance_type_family string, savings_plan_net_amortized_upfront_commitment_for_billing_period double, savings_plan_net_recurring_commitment_for_billing_period double, savings_plan_net_savings_plan_effective_cost double, savings_plan_offering_type string, savings_plan_payment_option string, savings_plan_purchase_term string, savings_plan_recurring_commitment_for_billing_period double, savings_plan_region string, savings_plan_savings_plan_a_r_n string, savings_plan_savings_plan_effective_cost double, savings_plan_savings_plan_rate double, savings_plan_start_time string, savings_plan_total_commitment_to_date double, savings_plan_used_commitment double, split_line_item_actual_usage double, split_line_item_net_split_cost double, split_line_item_net_unused_cost double, split_line_item_parent_resource_id string, split_line_item_public_on_demand_split_cost double, split_line_item_public_on_demand_unused_cost double, split_line_item_reserved_usage double, split_line_item_split_cost double, split_line_item_split_usage double, split_line_item_split_usage_ratio double, split_line_item_unused_cost double, tags map<string,string> ) PARTITIONED BY (billing_period string) STORED AS PARQUET LOCATION 's3://wiv-cur2-123456789012/wiv/Wiv-CUR2/data/' TBLPROPERTIES ( 'classification'='parquet', 'compressionType'='snappy', 'parquet.compression'='SNAPPY', 'projection.enabled'='true', 'projection.billing_period.type'='date', 'projection.billing_period.format'='yyyy-MM', 'projection.billing_period.range'='2020-01,NOW', 'projection.billing_period.interval'='1', 'projection.billing_period.interval.unit'='MONTHS', 'storage.location.template'='s3://wiv-cur2-123456789012/wiv/Wiv-CUR2/data/BILLING_PERIOD=${billing_period}' )

Step 8: Add permissions to WivAccessRole

Edit the WivPayerAccessPolicy policy with the new bucket ARN

{ "Effect": "Allow", "Action": "s3:*", "Resource": [ "arn:aws:s3:::wiv-cur-123456789012", "arn:aws:s3:::wiv-cur-123456789012/*", "arn:aws:s3:::wiv-cur2-123456789012", "arn:aws:s3:::wiv-cur2-123456789012/*" ] }

Step 9: Test Your Setup

Basic test:

SELECT * FROM wivdb.wiv_cur2 WHERE billing_period = '2025-12' LIMIT 10

Cost by service:

SELECT line_item_product_code AS service, SUM(line_item_unblended_cost) AS cost FROM wivdb.wiv_cur2 WHERE billing_period = '2025-12' GROUP BY line_item_product_code ORDER BY cost DESC LIMIT 10

Cost by account:

SELECT line_item_usage_account_id AS account_id, line_item_usage_account_name AS account_name, SUM(line_item_unblended_cost) AS cost FROM wivdb.wiv_cur2 WHERE billing_period = '2025-12' GROUP BY 1, 2 ORDER BY cost DESC

Summary

StepResourcePurpose
1S3 BucketStore CUR data
2Bucket PolicyAllow AWS to write CUR
3CUR ExportGenerate CUR 2.0 data
4WaitFirst delivery ~24 hours
5Glue DatabaseCatalog for Athena
6Athena WorkgroupQuery workspace
7Athena TableQuery CUR with partition projection
8TestVerify setup

S3 Data Structure

s3://wiv-cur-{account}/ └── wiv/ └── Wiv-CUR/ ├── data/ │ ├── BILLING_PERIOD=2025-11/ │ │ └── Wiv-CUR-00001.snappy.parquet │ └── BILLING_PERIOD=2025-12/ │ └── Wiv-CUR-00001.snappy.parquet └── metadata/ └── BILLING_PERIOD=2025-12/ └── Wiv-CUR-Manifest.json

Troubleshooting

No data in S3

  • Wait up to 24 hours for first delivery
  • Check export status in Billing → Data Exports

Query returns no results

  • Verify S3 path matches table LOCATION
  • Check billing_period format matches folder name (YYYY-MM)

Type mismatch errors

  • Use the exact schema provided above
  • Key: savings_plan_savings_plan_rate must be double

Partition projection not working

  • Ensure all TBLPROPERTIES are set correctly
  • The storage.location.template must match actual S3 path structure

Notes

  • No Glue Crawler needed - Partition projection handles this automatically
  • No MSCK REPAIR TABLE needed - New months are discovered automatically
  • Glue shows 0 partitions - This is normal with partition projection
  • Multi-account: If using AWS Organizations, management account CUR includes all member accounts