[Tools] Add documents for tools script; Add NG for tools

tools/docs/README.md

@@ -0,0 +1,162 @@
+# RT-Thread 构建系统文档
+
+欢迎使用RT-Thread构建系统文档。本文档集详细介绍了RT-Thread基于SCons的构建系统的使用方法和技术原理。
+
+## 文档目录
+
+### 📚 用户指南
+
+1. **[构建系统使用指南](构建系统使用指南.md)**
+   - 快速开始
+   - 命令行选项详解
+   - 工具链配置
+   - 项目生成
+   - 软件包管理
+   - 高级功能
+   - 常见问题解答
+
+2. **[SConscript编写指南](SConscript编写指南.md)**
+   - 基础语法
+   - 常用模式
+   - 高级技巧
+   - 最佳实践
+   - 示例集合
+
+### 🔧 技术文档
+
+3. **[构建系统技术原理](构建系统技术原理.md)**
+   - 系统架构设计
+   - 核心模块分析
+   - 构建流程详解
+   - 依赖管理机制
+   - 工具链适配层
+   - 项目生成器架构
+   - 扩展机制
+
+## 快速导航
+
+### 常用命令
+
+```bash
+# 基础编译
+scons                    # 默认编译
+scons -j8               # 8线程并行编译
+scons -c                # 清理编译产物
+
+# 配置管理
+menuconfig              # 图形化配置
+scons --pyconfig       # Python脚本配置
+
+# 项目生成
+scons --target=mdk5    # 生成Keil MDK5项目
+scons --target=iar     # 生成IAR项目
+scons --target=vsc     # 生成VS Code项目
+scons --target=cmake   # 生成CMake项目
+
+# 软件包管理
+pkgs --update          # 更新软件包
+pkgs --list           # 列出已安装包
+```
+
+### 核心概念
+
+- **SConstruct**: BSP根目录的主构建脚本
+- **SConscript**: 各个组件/目录的构建脚本
+- **rtconfig.py**: 工具链和平台配置
+- **rtconfig.h**: RT-Thread功能配置
+- **DefineGroup**: 定义组件的核心函数
+- **GetDepend**: 检查依赖的核心函数
+
+## 构建系统架构图
+
+```
+┌─────────────────────────────────────┐
+│          用户命令 (scons)           │
+└──────────────┬──────────────────────┘
+               │
+┌──────────────▼──────────────────────┐
+│         SConstruct (主脚本)         │
+│     ┌─────────────────────┐        │
+│     │   PrepareBuilding()  │        │
+│     │   环境初始化         │        │
+│     └──────────┬──────────┘        │
+└────────────────┼───────────────────┘
+                 │
+┌────────────────▼────────────────────┐
+│         building.py                 │
+│   ┌──────────┬──────────┐          │
+│   │ 组件收集 │ 依赖处理 │          │
+│   └──────────┴──────────┘          │
+└─────────────────────────────────────┘
+                 │
+        ┌────────┴────────┐
+        │                 │
+┌───────▼──────┐ ┌────────▼────────┐
+│  SConscript  │ │   rtconfig.h    │
+│  组件脚本    │ │   功能配置      │
+└──────────────┘ └─────────────────┘
+```
+
+## 主要特性
+
+✅ **多工具链支持**
+- GCC (ARM/RISC-V/x86)
+- Keil MDK (ARMCC/ARMClang)
+- IAR
+- Visual Studio
+
+✅ **灵活的配置系统**
+- Kconfig图形配置
+- 条件编译支持
+- 本地编译选项
+
+✅ **丰富的项目生成器**
+- IDE项目文件生成
+- CMake支持
+- Makefile生成
+- VS Code配置
+
+✅ **模块化设计**
+- 组件独立构建
+- 清晰的依赖管理
+- 可扩展架构
+
+## 开发工作流
+
+```mermaid
+graph LR
+    A[配置系统] --> B[编写代码]
+    B --> C[构建项目]
+    C --> D[调试运行]
+    D --> E{是否完成?}
+    E -->|否| B
+    E -->|是| F[发布]
+    
+    A1[menuconfig] -.-> A
+    C1[scons] -.-> C
+    C2[IDE项目] -.-> C
+```
+
+## 相关链接
+
+- [RT-Thread官网](https://www.rt-thread.org)
+- [RT-Thread GitHub](https://github.com/RT-Thread/rt-thread)
+- [SCons官方文档](https://scons.org/documentation.html)
+
+## 贡献指南
+
+如果您发现文档中的错误或有改进建议，欢迎：
+
+1. 在GitHub上提交Issue
+2. 提交Pull Request
+3. 在RT-Thread社区论坛反馈
+
+## 版本信息
+
+- 文档版本：1.0.0
+- 更新日期：2024-01
+- 适用版本：RT-Thread 4.1.0+
+
+---
+
+**注意**：本文档基于RT-Thread最新版本编写，部分功能可能需要特定版本支持。使用前请确认您的RT-Thread版本。

tools/ng/__init__.py

@@ -0,0 +1,25 @@
+# -*- coding: utf-8 -*-
+"""
+RT-Thread Next Generation Build System
+
+This module provides an object-oriented implementation of the RT-Thread build system
+while maintaining backward compatibility with the existing building.py interface.
+"""
+
+from .core import BuildContext
+from .environment import RTEnv
+from .config import ConfigManager
+from .project import ProjectRegistry, ProjectGroup
+from .toolchain import ToolchainManager
+from .generator import GeneratorRegistry
+
+__version__ = "1.0.0"
+__all__ = [
+    'BuildContext',
+    'RTEnv', 
+    'ConfigManager',
+    'ProjectRegistry',
+    'ProjectGroup',
+    'ToolchainManager',
+    'GeneratorRegistry'
+]

tools/ng/building_ng.py

@@ -0,0 +1,115 @@
+# -*- coding: utf-8 -*-
+"""
+Next Generation building.py with minimal modifications.
+
+This file shows how to integrate the new OOP system with minimal changes to building.py.
+The actual implementation would modify the original building.py file.
+"""
+
+# Import everything from original building.py
+import sys
+import os
+
+# Add parent directory to path to import original building
+sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
+from building import *
+
+# Import new OOP modules
+from ng.adapter import (
+    init_build_context,
+    inject_environment_methods,
+    load_rtconfig as ng_load_rtconfig,
+    GenerateProject as ng_GenerateProject
+)
+
+
+# Override PrepareBuilding to integrate new system
+_original_PrepareBuilding = PrepareBuilding
+
+def PrepareBuilding(env, root_directory, has_libcpu=False, remove_components=[]):
+    """
+    Enhanced PrepareBuilding that integrates the new OOP system.
+    
+    This function wraps the original PrepareBuilding and adds OOP functionality.
+    """
+    # Initialize new build context
+    context = init_build_context(root_directory)
+    
+    # Call original PrepareBuilding
+    result = _original_PrepareBuilding(env, root_directory, has_libcpu, remove_components)
+    
+    # Inject new methods into environment
+    inject_environment_methods(env)
+    
+    # Load configuration into new system
+    ng_load_rtconfig('rtconfig.h')
+    
+    # Store context in environment for access
+    env['_BuildContext'] = context
+    
+    return result
+
+
+# Override DefineGroup to use new implementation
+_original_DefineGroup = DefineGroup
+
+def DefineGroup(name, src, depend, **parameters):
+    """
+    Enhanced DefineGroup that uses the new OOP implementation.
+    
+    This maintains backward compatibility while using the new system internally.
+    """
+    # Get environment from global Env
+    global Env
+    if Env and hasattr(Env, 'DefineGroup'):
+        # Use new method if available
+        return Env.DefineGroup(name, src, depend, **parameters)
+    else:
+        # Fallback to original
+        return _original_DefineGroup(name, src, depend, **parameters)
+
+
+# Override GetDepend to use new implementation
+_original_GetDepend = GetDepend
+
+def GetDepend(depend):
+    """
+    Enhanced GetDepend that uses the new OOP implementation.
+    """
+    global Env
+    if Env and hasattr(Env, 'GetDepend'):
+        # Use new method if available
+        return Env.GetDepend(depend)
+    else:
+        # Fallback to original
+        return _original_GetDepend(depend)
+
+
+# Override DoBuilding to integrate project generation
+_original_DoBuilding = DoBuilding
+
+def DoBuilding(target, objects):
+    """
+    Enhanced DoBuilding that integrates new project generation.
+    """
+    # Call original DoBuilding
+    _original_DoBuilding(target, objects)
+    
+    # Handle project generation with new system
+    if GetOption('target'):
+        target_name = GetOption('target')
+        global Env, Projects
+        
+        # Use new generator if available
+        try:
+            ng_GenerateProject(target_name, Env, Projects)
+        except Exception as e:
+            print(f"Falling back to original generator: {e}")
+            # Call original GenTargetProject
+            from building import GenTargetProject
+            GenTargetProject(Projects, program=target)
+
+
+# Export enhanced functions
+__all__ = ['PrepareBuilding', 'DefineGroup', 'GetDepend', 'DoBuilding'] + \
+          [name for name in dir(sys.modules['building']) if not name.startswith('_')]

tools/ng/core.py

@@ -0,0 +1,176 @@
+# -*- coding: utf-8 -*-
+"""
+Core module for RT-Thread build system.
+
+This module provides the central BuildContext class that manages the build state
+and coordinates between different components.
+"""
+
+import os
+import logging
+from typing import Dict, List, Optional, Any
+from dataclasses import dataclass, field
+
+from .config import ConfigManager
+from .project import ProjectRegistry
+from .toolchain import ToolchainManager
+from .generator import GeneratorRegistry
+from .utils import PathService
+
+
+class BuildContext:
+    """
+    Central build context that manages all build-related state.
+    
+    This class replaces the global variables in building.py with a proper
+    object-oriented design while maintaining compatibility.
+    """
+    
+    # Class variable to store the current context (for backward compatibility)
+    _current_context: Optional['BuildContext'] = None
+    
+    def __init__(self, root_directory: str):
+        """
+        Initialize build context.
+        
+        Args:
+            root_directory: RT-Thread root directory path
+        """
+        self.root_directory = os.path.abspath(root_directory)
+        self.bsp_directory = os.getcwd()
+        
+        # Initialize managers
+        self.config_manager = ConfigManager()
+        self.project_registry = ProjectRegistry()
+        self.toolchain_manager = ToolchainManager()
+        self.generator_registry = GeneratorRegistry()
+        self.path_service = PathService(self.bsp_directory)
+        
+        # Build environment
+        self.environment = None
+        self.build_options = {}
+        
+        # Logging
+        self.logger = self._setup_logger()
+        
+        # Set as current context
+        BuildContext._current_context = self
+        
+    @classmethod
+    def get_current(cls) -> Optional['BuildContext']:
+        """Get the current build context."""
+        return cls._current_context
+        
+    @classmethod
+    def set_current(cls, context: Optional['BuildContext']) -> None:
+        """Set the current build context."""
+        cls._current_context = context
+        
+    def _setup_logger(self) -> logging.Logger:
+        """Setup logger for build system."""
+        logger = logging.getLogger('rtthread.build')
+        if not logger.handlers:
+            handler = logging.StreamHandler()
+            formatter = logging.Formatter('[%(levelname)s] %(message)s')
+            handler.setFormatter(formatter)
+            logger.addHandler(handler)
+            logger.setLevel(logging.INFO)
+        return logger
+        
+    def prepare_environment(self, env) -> None:
+        """
+        Prepare the build environment.
+        
+        Args:
+            env: SCons Environment object
+        """
+        self.environment = env
+        
+        # Set environment variables
+        env['RTT_ROOT'] = self.root_directory
+        env['BSP_ROOT'] = self.bsp_directory
+        
+        # Add to Python path
+        import sys
+        tools_path = os.path.join(self.root_directory, 'tools')
+        if tools_path not in sys.path:
+            sys.path.insert(0, tools_path)
+            
+        self.logger.debug(f"Prepared environment with RTT_ROOT={self.root_directory}")
+        
+    def load_configuration(self, config_file: str = 'rtconfig.h') -> None:
+        """
+        Load configuration from rtconfig.h.
+        
+        Args:
+            config_file: Path to configuration file
+        """
+        config_path = os.path.join(self.bsp_directory, config_file)
+        if os.path.exists(config_path):
+            self.config_manager.load_from_file(config_path)
+            self.build_options = self.config_manager.get_all_options()
+            self.logger.info(f"Loaded configuration from {config_file}")
+        else:
+            self.logger.warning(f"Configuration file {config_file} not found")
+            
+    def get_dependency(self, depend: Any) -> bool:
+        """
+        Check if dependency is satisfied.
+        
+        Args:
+            depend: Dependency name or list of names
+            
+        Returns:
+            True if dependency is satisfied
+        """
+        return self.config_manager.get_dependency(depend)
+        
+    def register_project_group(self, group) -> None:
+        """
+        Register a project group.
+        
+        Args:
+            group: ProjectGroup instance
+        """
+        self.project_registry.register_group(group)
+        
+    def merge_groups(self) -> List:
+        """
+        Merge all registered project groups.
+        
+        Returns:
+            List of build objects
+        """
+        return self.project_registry.merge_groups(self.environment)
+
+
+@dataclass
+class BuildOptions:
+    """Build options container."""
+    verbose: bool = False
+    strict: bool = False
+    target: Optional[str] = None
+    jobs: int = 1
+    clean: bool = False
+    
+    
+@dataclass 
+class ProjectInfo:
+    """Project information for generators."""
+    name: str = "rtthread"
+    target_name: str = "rtthread.elf"
+    
+    # File collections
+    source_files: List[str] = field(default_factory=list)
+    include_paths: List[str] = field(default_factory=list)
+    defines: Dict[str, str] = field(default_factory=dict)
+    
+    # Compiler options
+    cflags: str = ""
+    cxxflags: str = ""
+    asflags: str = ""
+    ldflags: str = ""
+    
+    # Libraries
+    libs: List[str] = field(default_factory=list)
+    lib_paths: List[str] = field(default_factory=list)

tools/ng/integration_example.py

@@ -0,0 +1,178 @@
+# -*- coding: utf-8 -*-
+"""
+Example of minimal changes needed in building.py to integrate the new OOP system.
+
+This file shows the exact changes that would be made to the original building.py.
+"""
+
+# =============================================================================
+# CHANGES TO ADD AT THE BEGINNING OF building.py
+# =============================================================================
+
+"""
+# Add after the imports section in building.py (around line 45)
+
+# Try to import new OOP system
+try:
+    from ng.adapter import (
+        init_build_context,
+        inject_environment_methods,
+        load_rtconfig as ng_load_rtconfig,
+        MergeGroups as ng_MergeGroups
+    )
+    NG_AVAILABLE = True
+except ImportError:
+    NG_AVAILABLE = False
+"""
+
+# =============================================================================
+# CHANGES IN PrepareBuilding FUNCTION
+# =============================================================================
+
+"""
+# Add these lines in PrepareBuilding function after setting up Env (around line 70)
+
+    # Initialize new OOP system if available
+    if NG_AVAILABLE:
+        # Initialize build context
+        ng_context = init_build_context(Rtt_Root)
+        
+        # Inject methods into environment
+        inject_environment_methods(Env)
+        
+        # Store context reference
+        Env['__NG_Context'] = ng_context
+"""
+
+# =============================================================================
+# CHANGES AFTER PARSING rtconfig.h
+# =============================================================================
+
+"""
+# Add after parsing rtconfig.h (around line 430)
+
+    # Load configuration into new system
+    if NG_AVAILABLE and 'rtconfig.h' in os.listdir(Bsp_Root):
+        ng_load_rtconfig('rtconfig.h')
+"""
+
+# =============================================================================
+# ENHANCED DefineGroup FUNCTION
+# =============================================================================
+
+"""
+# Replace the original DefineGroup function (around line 565) with:
+
+def DefineGroup(name, src, depend, **parameters):
+    global Env
+    if Env is None:
+        return []
+    
+    # Try to use new implementation if available
+    if NG_AVAILABLE and hasattr(Env, 'DefineGroup'):
+        return Env.DefineGroup(name, src, depend, **parameters)
+    
+    # Original implementation continues below...
+    # [Keep all the original DefineGroup code here]
+"""
+
+# =============================================================================
+# ENHANCED GetDepend FUNCTION  
+# =============================================================================
+
+"""
+# Replace the original GetDepend function (around line 655) with:
+
+def GetDepend(depend):
+    global Env
+    
+    # Try to use new implementation if available
+    if NG_AVAILABLE and Env and hasattr(Env, 'GetDepend'):
+        return Env.GetDepend(depend)
+    
+    # Original implementation continues below...
+    # [Keep all the original GetDepend code here]
+"""
+
+# =============================================================================
+# ENHANCED MergeGroup FUNCTION
+# =============================================================================
+
+"""
+# Replace the original MergeGroup function (around line 700) with:
+
+def MergeGroup(src_group, group):
+    # Try to use new implementation if available
+    if NG_AVAILABLE and Env and hasattr(Env, '__NG_Context'):
+        context = Env['__NG_Context']
+        if context:
+            # Register groups with new system
+            from ng.project import ProjectGroup
+            for g in group:
+                if 'name' in g:
+                    pg = ProjectGroup(
+                        name=g['name'],
+                        sources=g.get('src', []),
+                        dependencies=[],
+                        environment=Env
+                    )
+                    context.register_project_group(pg)
+    
+    # Original implementation continues below...
+    # [Keep all the original MergeGroup code here]
+"""
+
+# =============================================================================
+# EXAMPLE USAGE IN SCONSCRIPT
+# =============================================================================
+
+def example_sconscript():
+    """
+    Example of how to use the new features in a SConscript file.
+    """
+    sconscript_content = '''
+from building import *
+
+# Get environment
+env = GetEnvironment()
+
+# Method 1: Use new environment methods (if available)
+if hasattr(env, 'DefineGroup'):
+    # New OOP style
+    src = env.GlobFiles('*.c')
+    group = env.DefineGroup('MyComponent', src, depend=['RT_USING_XXX'])
+else:
+    # Fallback to traditional style
+    src = Glob('*.c')
+    group = DefineGroup('MyComponent', src, depend=['RT_USING_XXX'])
+
+# Method 2: Always compatible style
+src = Glob('*.c')
+group = DefineGroup('MyComponent', src, depend=['RT_USING_XXX'])
+
+Return('group')
+'''
+    return sconscript_content
+
+# =============================================================================
+# MINIMAL CHANGES SUMMARY
+# =============================================================================
+
+"""
+Summary of changes needed in building.py:
+
+1. Add imports at the beginning (5 lines)
+2. Add initialization in PrepareBuilding (6 lines)  
+3. Add config loading after rtconfig.h parsing (3 lines)
+4. Modify DefineGroup to check for new method (3 lines)
+5. Modify GetDepend to check for new method (3 lines)
+6. Enhance MergeGroup to register with new system (15 lines)
+
+Total: ~35 lines of code added/modified in building.py
+
+Benefits:
+- Fully backward compatible
+- Opt-in design (works even if ng module is not present)
+- Gradual migration path
+- No changes needed in existing SConscript files
+"""