autodocAI.md

@@ -65,7 +65,7 @@ This document describes the procedure for automated documentation generation usi
 
 Automation scripts were used for documentation generation in two different scenarios: one providing documentation for code and another offering an overview of functionalities based on modules. Below is the explanation of the operation of each.
 
-# Automated Documentation Generation Script
+# Automated CODE Documentation Generation Script
 
 ## Overview
 
@@ -161,3 +161,99 @@ To use the script, run the following command, replacing `<folder_name>` with the
 ```bash
 ./script_name.sh <folder_name>
 ```
+
+
+# Automated OVERVIEW Documentation Generation Script
+
+## Overview
+
+The `generate_docs.sh` script is designed to automate the generation of high-level documentation for a codebase. It uses the Bito CLI tool to generate textual documentation and Mermaid diagrams for visual representation. The script handles the creation of individual module documents, generates an overview diagram, and aggregates documentation into a comprehensive file.
+
+## Prerequisites
+
+1. **Bito CLI** - A tool for generating documentation.
+   - **Installation**:
+     - **macOS/Linux**: `sudo curl https://alpha.bito.ai/downloads/cli/install.sh -fsSL | bash`
+     - **Arch Linux**: `yay -S bito-cli` or `paru -S bito-cli`
+     - **Windows**: Download the MSI from Bito's website.
+2. **Mermaid CLI** - A tool for generating Mermaid diagrams.
+   - **Installation**: `npm install -g @mermaid-js/mermaid-cli`
+
+## Variables
+
+- `BITO_CMD`: Path to the Bito CLI command.
+- `BITO_CMD_VEP`: Optional flag for Bito CLI based on version.
+- `log_file`: Log file for token usage information.
+- `prompt_folder`: Directory containing prompt files for NLP tasks.
+- `total_input_token_count`: Total input tokens used in the session.
+- `total_output_token_count`: Total output tokens generated in the session.
+- `lang_csv`: CSV file with programming language extensions.
+- `skip_list_csv`: CSV file containing paths to skip.
+
+## Functions
+
+### `bito_response_ok()`
+Checks if the response from Bito CLI is valid. Returns `0` if valid, `1` otherwise.
+
+### `update_token_usage()`
+Updates the global token usage counters based on input and output text.
+
+### `log_token_usage_and_session_duration()`
+Logs the total token usage and session duration to the log file.
+
+### `check_tools_and_files()`
+Checks for the presence of required tools and files, exits with an error if any are missing.
+
+### `read_skip_list()`
+Reads the skip list from a CSV file to determine which paths should be excluded.
+
+### `is_skippable()`
+Determines if a path should be skipped based on predefined patterns.
+
+### `call_bito_with_retry()`
+Calls the Bito CLI with retry logic in case of temporary failures.
+
+### `create_module_documentation()`
+Generates documentation for an individual module, including Mermaid diagrams.
+
+### `extract_module_names_and_associated_objectives_then_call_bito()`
+Extracts module names and objectives from a file and generates documentation using Bito.
+
+### `fix_mermaid_syntax()`
+Fixes syntax issues in Mermaid diagrams.
+
+### `fix_mermaid_syntax_with_bito()`
+Uses Bito to fix Mermaid syntax issues.
+
+### `validate_mermaid_syntax()`
+Validates the syntax of a Mermaid diagram.
+
+### `fix_and_validate_mermaid()`
+Applies syntax fixes and validates Mermaid diagrams, optionally using Bito for additional fixing.
+
+### `generate_mermaid_diagram()`
+Generates Mermaid diagrams from markdown files.
+
+### `create_mermaid_diagram()`
+Creates Mermaid diagrams for a module with retry logic and syntax validation.
+
+### `generate_mdd_overview()`
+Generates an overview Mermaid diagram by combining all `.mdd` files in the specified directory.
+
+### `create_find_command()`
+Creates a `find` command to locate files with specific extensions for documentation.
+
+## Execution Flow
+
+1. **Check Tools and Files**: Ensures all required tools and files are available.
+2. **Folder Validation**: Verifies that the provided folder exists.
+3. **Documentation Generation**:
+   - Creates documentation for each module file found.
+   - Aggregates individual module documents into a main file.
+4. **Overview Generation**: Generates a comprehensive overview diagram of the system.
+5. **System Introduction and Summary**: Extracts and prepends an introduction and summary to the aggregated documentation.
+
+## Usage
+
+```bash
+./generate_docs.sh <folder_to_document>