This is the third piece in a series on identity and access management, and it's about the part of IAM nobody wants to own: policy documentation. In a fast-moving Okta environment, keeping documentation in sync with reality by hand is close to impossible. I built a pipeline that pulls policy data straight from Okta's API, turns it into Mermaid diagrams, and lets me query the whole thing in plain English instead of tracing through the admin console rule by rule.
The case for automated documentation
Manually tracked policy documentation is a mess by default — scattered notes, docs that drift out of sync the moment someone edits a rule in the console, no reliable way to know if what's written down still matches reality. Automating it isn't really about saving time, though that's a real side benefit. It's about accuracy: the documentation reflects the actual environment because it's generated directly from the actual environment. It's about having one source of truth instead of five people's notes. And it scales the same way regardless of how large the policy set gets, because the process doesn't get more tedious as the environment grows — only the runtime does.
Pulling policy data out of Okta's API
Okta's Policy API is where this starts. It's the only reliable source for what's actually configured, across every policy type — sign-on, password, MFA enrollment, profile enrollment, and the rest. Here's the fetch:
import os
import sys
import json
import requests
import pandas as pd
from dotenv import load_dotenv
load_dotenv()
okta_api_token = os.getenv("OKTA_API_TOKEN")
okta_org_url = os.getenv("OKTA_ORG_URL")
headers = {
"Accept": "application/json",
"Content-Type": "application/json",
"Authorization": okta_api_token
}
policy_types = ['ACCESS_POLICY', 'IDP_DISCOVERY', 'MFA_ENROLL',
'OKTA_SIGN_ON', 'PASSWORD', 'PROFILE_ENROLLMENT']
## Get all Okta Policies
def fetch_policies_by_type(policy_type):
url = f"{okta_org_url}policies?type={policy_type}"
response = requests.get(url, headers=headers)
if response.status_code == 200:
return response.json()
else:
print(f"Error fetching policies of type {policy_type}")
print(response.status_code)
return None
def fetch_all_policies():
policies = []
for policy_type in policy_types:
policies.extend(fetch_policies_by_type(policy_type))
return pd.DataFrame(policies)
df_okta_policies = fetch_all_policies()
The shape of a policy object
Before this data is useful for anything, it helps to understand the actual structure of a policy and the rules attached to it. Each policy object is a repository of detail, and getting the model right up front is what makes the diagrams and the AI querying downstream correct instead of just plausible-looking.
Attributes of a policy object:
- ID — the unique identifier, needed to associate rules to the right policy
- Type and name — type buckets it (Sign-On, Password, MFA); name is the human-readable label
- Conditions and actions — conditions determine when a policy applies, actions determine what happens when it does. Both can nest several layers deep depending on the policy type.
Attributes of a rule:
- Rule ID — same idea as the policy ID, links a rule back to its parent
- Priority — rules evaluate in priority order, which matters whenever more than one rule could match the same request
- Conditions — more granular than the policy-level conditions: user group, network zone, device compliance, and so on
- Actions — the actual enforcement: prompt for MFA, set a session lifetime, deny outright
Policies set the overall framework; rules are what actually gets evaluated against an incoming request. That relationship is the whole point of getting the data model right before touching any visualization code.
Turning the data into diagrams
Raw JSON in a DataFrame isn't something anyone wants to read directly. Mermaid turns it into a diagram — every rule shown as a node, color-coded by whether it allows or denies access, connected back to its parent policy.
First, the policy data gets reshaped into a format Mermaid can actually consume:
#Create function to fetch rules using the provided rules link
def fetch_rules(rules_link):
response = requests.get(rules_link, headers=headers)
rules = response.json()
return pd.DataFrame(rules)
# Create function to transform rules, conditions, and actions
def transform_rules(rules_df):
conditions_data = []
actions_data = []
for _, rule in rules_df.iterrows():
policy_id = rule['policy_id']
conditions_data.append({'policy_id': policy_id,
'Conditions': rule.get('conditions', {})})
actions_data.append({'policy_id': policy_id,
'Actions': rule.get('actions', {})})
return pd.DataFrame(conditions_data), pd.DataFrame(actions_data)
# Create functions to flatten the conditions and actions dataframes
def flatten_conditions(rule_id, policy_id, conditions):
if conditions is None:
return pd.DataFrame(columns=['rule_id', 'policy_id'])
else:
flattened_conditions = pd.json_normalize(conditions)
flattened_conditions['rule_id'] = rule_id
flattened_conditions['policy_id'] = policy_id
return flattened_conditions
def flatten_actions(rule_id, policy_id, actions):
if actions is None:
return pd.DataFrame(columns=['rule_id', 'policy_id'])
else:
flattened_actions = pd.json_normalize(actions)
flattened_actions['rule_id'] = rule_id
flattened_actions['policy_id'] = policy_id
return flattened_actions
df_rules = pd.DataFrame()
df_rule_conditions = pd.DataFrame()
df_rule_actions = pd.DataFrame()
for _, policy in df_okta_policies.iterrows():
rules_link = policy['_links']['rules']['href']
policy_rules_df = fetch_rules(rules_link)
policy_rules_df['policy_id'] = policy['id']
# Adding policy_id as foreign key
for _, rule in policy_rules_df.iterrows():
try:
conditions_data = flatten_conditions(rule['id'], policy['id'],
rule['conditions'])
actions_data = flatten_actions(rule['id'], policy['id'],
rule['actions'])
except Exception as e:
print(f"Error: {type(e).__name__}")
if str(e):
print(f"Error message: {e}")
# Concatenate data to the respective DataFrame
df_rule_conditions = pd.concat([df_rule_conditions,
conditions_data], ignore_index=True)
df_rule_actions = pd.concat([df_rule_actions,
actions_data], ignore_index=True)
df_rules = pd.concat([df_rules, policy_rules_df], ignore_index=True)
# Rename columns to allow the llms to play nicely
df_okta_policies.rename(columns={'id': 'policy_id', 'name': 'policy_name',
'status': 'policy_status',
'type': 'policy_type', 'conditions': 'policy_conditions'},
inplace=True)
df_okta_policies.head()
Renaming columns for consistency and wiring the policy and rule IDs through every table is what makes it possible to join everything back together later — for the diagrams, and for the AI querying that comes after:
# Rename our columns for consistency
df_rules.rename(columns={'id': 'rule_id', 'type': 'policy_type',
'conditions': 'rule_conditions', 'actions': 'rule_actions',
'name': 'rule_name', 'status': 'rule_status'}, inplace=True)
## add rule_name and policy_name to df_rule_conditions dataframe
df_rule_conditions = pd.merge(df_rule_conditions,
df_rules[['rule_id', 'rule_name']],
on='rule_id', how='left')
df_rule_conditions = pd.merge(df_rule_conditions,
df_okta_policies[['policy_id', 'policy_name']],
on='policy_id', how='left')
# add rule_name and policy_name to df_rule_actions dataframe
df_rule_actions = pd.merge(df_rule_actions,
df_rules[['rule_id', 'rule_name']],
on='rule_id', how='left')
df_rule_actions = pd.merge(df_rule_actions,
df_okta_policies[['policy_id', 'policy_name']],
on='policy_id', how='left')
For this pass I scoped it down to just the access policies tied to applications — that's the set that actually matters most for day-to-day audit questions:
# Get the applications linked to a policy using the Okta API
# Example: https://{yourOktaDomain}/api/v1/policies/{policyId}/mappings
# Create a dataset with the Access Policy Type
df_access_policies = df_okta_policies[df_okta_policies['policy_type'] == 'ACCESS_POLICY']
# Iterate through each policy and create a dataset with the application
# and policy context
df_okta_applications = pd.DataFrame()
for _, policy in df_access_policies.iterrows():
policy_id = policy['policy_id']
url = f"{okta_org_url}policies/{policy_id}/mappings"
response = requests.get(url, headers=headers)
if response.status_code == 200:
applications = response.json()
for application in applications:
application['policy_id'] = policy_id
df_okta_applications = pd.concat([df_okta_applications,
pd.DataFrame(applications)],
ignore_index=True)
else:
print(f"Error fetching applications for policy {policy_id}")
print(response.status_code)
df_okta_applications.head()
I'd rather reference applications by their label than a raw ID, so this pulls that in too — along with the policy ID, so each application maps cleanly back to the policy governing it:
# Create a function to get the label details of an application
def extract_application_name(links):
response = requests.get(links['application']['href'], headers=headers)
if response.status_code == 200:
application = response.json()
return application['label']
else:
print(f"Error fetching application")
print(response.status_code)
return None
df_okta_applications['application_name'] = df_okta_applications['_links'].apply(extract_application_name)
# Add Policy Name and ID content to the application dataframe
df_okta_applications = pd.merge(df_okta_applications,
df_okta_policies[['policy_id', 'policy_name']],
on='policy_id', how='left')
df_okta_applications.head()
MFA constraints can combine in enough different ways that they needed their own parsing function rather than being handled inline:
# Helper function to safely parse JSON strings in DataFrame
def safe_json_loads(s):
try:
return json.loads(s)
except ValueError:
return None
# Create a function to parse constraints
def parse_constraints(constraints):
if not isinstance(constraints, list) or not constraints:
return "None"
readable_constraints = []
for constraint in constraints:
parts = []
for factor_class, details in constraint.items():
detail_parts = [f"{key}: {', '.join(value) if isinstance(value, list) else value}"
for key, value in details.items()]
parts.append(f"{factor_class.capitalize()}({' ; '.join(detail_parts)})")
readable_constraints.append(' & '.join(parts))
# Remove any parentheses from the constraints string value
readable_constraints = [constraint.replace('(', ' ').replace(')', '') for constraint in readable_constraints]
return ' OR '.join(readable_constraints)
# GEt a function to get constraints from a series
def get_constraints(constraints_series):
constraints_values = "None"
# After dropna, check if the series is empty
non_null_constraints = constraints_series.dropna()
if not non_null_constraints.empty:
# Extract the first non-null constraint
constraints_data = non_null_constraints.iloc[0]
# Check if constraints_data is a string and parse it
if isinstance(constraints_data, str):
constraints_json = safe_json_loads(constraints_data)
elif pd.notna(constraints_data):
constraints_json = constraints_data # If already a list/dict
else:
constraints_json = None
# Generate constraints description
if constraints_json:
constraints_values = parse_constraints(constraints_json)
return constraints_values
This is the actual diagram generation — using Mermaid's class styling to color each rule red or green depending on whether it allows or denies access:
# Function which uses the dataframes to generate mermaid diagrams for each policy
def generate_md(policy, rules_df, actions_df, conditions_df):
policy_id = policy['policy_id']
policy_name = policy['policy_name']
# Format the 'Created' and 'Last Updated' dates
created_date = datetime.strptime(policy['created'], '%Y-%m-%dT%H:%M:%S.%fZ').strftime('%Y-%m-%d')
last_updated_date = datetime.strptime(policy['lastUpdated'], '%Y-%m-%dT%H:%M:%S.%fZ').strftime('%Y-%m-%d')
# Filter the rules DataFrame to get only the rules for the current policy
rules = rules_df[rules_df['policy_id'] == policy_id]
md_content = f""" ```mermaid
graph LR
classDef DENY fill:#B80F0A,stroke:#333,stroke-width:1px;
classDef ALLOW fill:#0B6623,stroke:#333,stroke-width:1px;
classDef POLICY fill:grey,stroke:#333,stroke-width:1px;
%%Policy%%
{policy_id}("Policy:{policy_name}"
Policy id: {policy_id}\\n
Created:{policy['created']}
Last Updated:{policy['lastUpdated']}
System Policy:{policy['system']}
)
"""
for _, rule in rules.iterrows():
rule_id = rule['rule_id']
rule_name = rule['rule_name']
rule_permission = 'DENY' if rule['rule_actions']['appSignOn']['access'] == 'DENY' else 'ALLOW'
rule_status = rule['rule_status']
# Filter the actions and conditions DataFrames to get only the actions and conditions for the current rule
actions = actions_df[actions_df['rule_id'] == rule_id]
conditions = conditions_df[conditions_df['rule_id'] == rule_id]
if conditions.empty:
platform_type = 'Any'
platform_os = 'Any'
risk_level = 'Any'
else:
# Replace NaN values with 'ANY'
platform_type = 'ANY' if pd.isna(conditions['platform.include']).any() else conditions['platform.include'].iloc[0][0].get('type', 'Any')
platform_os = 'ANY' if pd.isna(conditions['platform.include']).any() else conditions['platform.include'].iloc[0][0]['os'].get('type', 'Any')
risk_level = conditions['riskScore.level'].iloc[0]
constraints_values = get_constraints(actions_df[actions_df['rule_id'] == rule_id]['appSignOn.verificationMethod.constraints'])
factor_mode = actions['appSignOn.verificationMethod.factorMode'].iloc[0]
group_ids = conditions['people.groups.include']
# Flatten the list of lists into a single list
flat_group_ids = [item for sublist in group_ids if isinstance(sublist, list) for item in sublist]
# Join the group IDs into a string
group_ids_str = ', '.join(flat_group_ids) if flat_group_ids else 'Any'
md_content += f""" %%Rules%%
{rule_id}["Rule:{rule_name}"
Rule id: {rule_id}\\n
Created:{created_date}
Last Updated:{last_updated_date}
Status:{rule['rule_status']}
Priority: {rule['priority']}
System Rule: {rule['system']}<br>
Conditions
Group: {group_ids_str}
Access: {rule_permission}
Platform Type: {platform_type}
Platform OS: {platform_os}
Risk Score Level: {risk_level}
"""
if rule_permission == 'ALLOW':
md_content += f""" Action Factor Mode: {factor_mode}
Constraints: {constraints_values}
"""
md_content += f""" ]
{policy_id} --> {rule_id}:::{rule_permission}
"""
md_content += "```"
return md_content
# Creates a file using the markdown file extension
def write_to_file(policy_name, md_content):
with open(f"{policy_name}_documentation.md", 'w') as f:
f.write(md_content)
With everything in place, generating diagrams for every application access policy comes down to a loop:
# Dataframe that contains only Access Policies
df_access_policies = df_okta_policies[df_okta_policies['policy_type'] == 'ACCESS_POLICY']
for _, policy in df_access_policies.iterrows():
file_name = policy['policy_name'] + "_doc.md"
file_path = os.path.join('policy_docs', file_name)
md_content = generate_md(policy, df_rules, df_rule_actions, df_rule_conditions)
write_to_file(file_path, md_content)
Running this against a live tenant produced a diagram per access policy — clean enough to screenshot directly into a Slack thread during an audit instead of narrating the rule structure out loud.
Querying the policies with an LLM
Diagrams solve "what does this policy do." They don't solve "what's the answer to this specific question about my policies" without me tracing through the graph myself. That's where an LLM came in — treating the policy DataFrames as a dataset it could query in natural language instead of me writing a new pandas filter by hand every time a question came up.
The DataFrames built for the Mermaid diagrams turned out to be exactly the input this needed too, with no separate prep work required. I tried a couple of options before landing on PandasAI — it handles multiple datasets at once, and, important for this use case, has an enforce_privacy flag that keeps the actual row data out of the request payload sent to the model:
# Setup environment for us to use LLMs
# We can add this to or set up environment earlier in our project.
from openai import OpenAI
from pandasai import SmartDataframe
from pandasai.llm import OpenAI
from pandasai import SmartDatalake
# For our experiment will be using OpenAI Models
# You can specify your own model here.
# Refer to Pandas AI documentation for a list of supported models
llm = OpenAI(api_token=openai_api_token, org=openai_api_org)
# Pandas AI is basically an LLM wrapper around pandas;
# By enforcing privcay we will avoid sending any data to OpenAI.
df = SmartDatalake([df_okta_policies, df_rule_conditions, df_rule_actions,
df_okta_applications, df_access_policies],
config={"llm": llm, "enforce_privacy": True})
Questions like "what are the conditions for the default MFA policy" or "what are the actions for the high-priority sign-on rules" turn into an actual lookup instead of a manual trace through the console. Even loosely phrased questions came back accurate — the model doesn't need precisely worded queries to map onto the right column, which matters more than it sounds like it should when you're asking it questions at the end of a long day.
The real value showed up in the relationships I wouldn't have thought to query for directly — asking a broad question and getting back a connection between two policies I hadn't noticed were related in the first place.
What still needs work: cleaning up how results get formatted, and building prompt templates so I don't have to specify which DataFrame to query against every single time.
Where this actually gets used
This is exactly the scenario it's built for. An auditor flags a question about remote-work sign-on policies, and instead of spending an hour combing through the console, I ask the LLM directly — "show me the sign-on rules for remote employees" — and get back the relevant policies with a Mermaid diagram and a plain-English breakdown in seconds. That turnaround is the entire reason to build this instead of just tolerating manual documentation indefinitely.
What's next
Combining API automation, Mermaid, and LLM querying turned policy documentation from a task everyone avoided into something I actually keep up to date — because keeping it up to date no longer takes real effort. Next up: using Terraform for change management on these same policies.
If you're dealing with the same documentation drift, the code above is a reasonable starting point. Pull your own policy data, adapt the Mermaid generation to your rule structure, and see where the LLM querying actually holds up versus where it needs more guardrails.