Skip to main content

Understanding Common Errors and Warnings

When defining your Data Vault model and settings using YAML files, S2V performs validation checks. Issues are reported as errors or warnings:

  • Errors (ERROR): Critical issues preventing artifact generation. Must be fixed.
  • Warnings (WARNING): Potential problems. Review and address recommended.

This document categorizes common messages.

1. File and Basic YAML Processing​

Checks for file presence and fundamental YAML syntax.

  • Missing Configuration Files:
    • ERROR: Missing information_schema.csv / data_vault_settings.yaml / source_system_settings.yaml
      • Fix: Ensure these files are in your input model's root.
  • YAML Syntax & Structure:
    • ERROR: YAML contains duplicated keys
      • Fix: Remove or rename duplicate keys in the YAML file.
    • ERROR: YAML parsing error (e.g., YAMLError, ParserError)
      • Fix: Check YAML syntax (indentation, colons) near the indicated line. See YAML Errors
    • ERROR: Empty or invalid YAML content
      • Fix: Ensure the YAML file has valid content and formatting.
    • WARNING: Ignored file: ..., only '.yaml' files can be processed
      • Fix (if needed): If the file should be processed, use a .yaml extension.
    • WARNING: YAML doesn't contain entity_type property and not a configuration file
      • Fix: Add the entity_type property (e.g., hub, link).
    • WARNING: Ignored file: ... Unknown entity_type in entity_type property
      • Fix: Correct the entity_type value.

2. Data Vault Model Definition​

Checks related to the overall structure and consistency of your Data Vault objects.

  • Object Naming & Uniqueness:
    • ERROR: DV model contains duplicated object [ObjectName] in files [FileA.yaml, FileB.yaml]
      • Fix: Ensure object names are unique across all YAML files. Rename or consolidate.
    • WARNING: The names of some objects in DV model differ only by case: [ObjectA (FileA.yaml), objecta (FileB.yaml)]
      • Fix: Use consistent casing or make names distinctly different to avoid confusion.
  • Partial Deployment:
    • ERROR: Entities [EntityName] do not exist. Make sure to provide existing objects for partial deployment.
      • Fix: If using --include-objects or --exclude-objects with generate or validate, ensure specified entity names exist in your model.

3. Data Vault Object YAML Structure (Schema Validation)​

This validation uses predefined schemas to check if your YAML files for Data Vault objects (Hubs, Links, Satellites, References) and global settings (data_vault_settings.yaml, source_system_settings.yaml) adhere to the expected structure, properties, data types, and patterns.

  • Common Schema Validation Message Format:
    • ERROR: YAML property: path -> to -> property - Message: [Detailed error message from schema validator]
    • The path -> to -> property points to the exact location of the issue in your YAML file.
  • Examples of Schema Validation Messages:
    • Message: 'property_name' is a required property
      • Meaning: A mandatory field is missing.
      • Fix: Add the required property. Refer to the specific model reference page (e.g., Hub, Link) for required fields.
    • Message: Additional properties are not allowed ('extra_prop' was unexpected)
      • Meaning: You've included a property that isn't defined for that object type or section.
      • Fix: Remove the unexpected property.
    • Message: 'value' is not of type 'integer'/'string'/'boolean'
      • Meaning: A property has the wrong data type.
      • Fix: Change the value to the correct data type.
    • Message: 'value' does not match 'pattern'
      • Meaning: A value doesn't follow a required format (e.g., URNs like urn:s2v:hub_source:NAME, or source table definitions like (schema,table)).
      • Fix: Adjust the value to match the expected pattern.

4. Data Vault Model Logic & Rules​

These checks validate the semantic correctness and adherence to Data Vault principles within your model definitions.

  • Hubs:
    • ERROR: Hub: [HubName] - Business key mapping for source [SourceURN] must not be empty.
      • Fix: Ensure business_key_mapping is defined for the specified entity source.
    • ERROR: Hub: [HubName] - Source business key for source [SourceURN] must not be empty because requires_source_business_key is true.
      • Fix: If requires_source_business_key: true on the Hub, provide a source_business_key value in the entity source.
    • ERROR: Hub: [HubName] - Source business key for source [SourceURN] must be empty because requires_source_business_key is false.
      • Fix: If requires_source_business_key: false on the Hub, ensure source_business_key is empty or omitted in the entity source.
  • Links:
    • ERROR: Link: [LinkName] - Connected hub [HubAlias -> HubName] does not exist.
      • Fix: Ensure all Hubs listed in connected_hubs are defined in your model.
    • ERROR: Link: [LinkName] - Connected hub relations for source [SourceURN] must not be empty.
      • Fix: Define connected_hub_relations for the entity source.
    • ERROR: Link: [LinkName] - Relation for connected hub [HubAlias] in source [SourceURN] is not defined.
      • Fix: Ensure each Hub alias in connected_hubs has a corresponding entry in connected_hub_relations for that source.
  • Satellites:
    • ERROR: Satellite: [SatName] - Connected entity [Hub/Link Name] does not exist.
      • Fix: Ensure the connected_entity (Hub or Link) is defined in your model.
    • ERROR: Link Satellite: [SatName] - Connected entity source reference [SourceURN] for connected link [LinkName] does not exist in the link's entity sources.
      • Fix: If connected_entity_source_ref is used, ensure the URN matches an existing source URN in the parent Link.
    • ERROR: Satellite: [SatName] - Business key mapping for source [SourceTuple] must not be empty.
      • Fix: Define business_key_mapping in the Satellite's entity_source.
  • General Entity Source Issues:
    • ERROR: Object: [ObjectName] - Entity source [SourceURN/Tuple] - Source system configuration URN [ConfigURN] does not exist in source_system_settings.yaml.
      • Fix: Ensure the source_system_configuration_urn points to a valid entry in source_system_settings.yaml.
    • ERROR: Object: [ObjectName] - Entity source [SourceURN/Tuple] - Business key mapping for target key [TargetBK] must not be empty.
      • Fix: Ensure all target business keys have source columns mapped to them.
    • ERROR: Object: [ObjectName] - Entity source [SourceURN/Tuple] - Lookup mapping for Hub source URN [HubSourceURN] does not exist in the Hub's entity sources.
      • Fix: If using lookup_mapping, ensure the hub_source_urn points to a valid source URN in the referenced Hub.

5. Information Schema and Source Column Validation​

Checks related to the consistency between your model definitions and the information_schema.csv.

  • ERROR: Column: [ColumnName] from table: [Schema.Table] is not defined in information_schema.csv.
    • Fix: Add the missing source column ColumnName for table Schema.Table to your information_schema.csv.
  • ERROR: Column: [ColumnName] from table: [Schema.Table] defined in information_schema.csv is not used in any DV object.
    • Fix: This is often a warning but can be an error. Remove the unused column from information_schema.csv or ensure it's correctly mapped in a DV object.
  • ERROR: Data type for column [ColumnName] from table [Schema.Table] is not defined in information_schema.csv.
    • Fix: Provide the data type for the specified column in information_schema.csv.

6. CLI Command Errors​

Errors specifically related to running S2V CLI commands (validate, generate, visualize).

  • s2v validate / s2v generate / s2v visualize:
    • ERROR: Invalid value for "-i" / "--input": Path "[Path]" does not exist.
      • Fix: Ensure the input path provided with the -i option is correct and accessible.
    • ERROR: Invalid value for "--data-vault-settings-path": Path "[Path]" does not exist. (and similar for other specific path arguments)
      • Fix: If overriding default file locations, ensure the specified path to the configuration file is correct.
    • ERROR: Authentication failed. Please check your credentials or token.
      • Fix: Ensure your S2V login credentials/token are valid and correctly configured. Run s2v login.
    • ERROR: Connection to Stream2Vault server failed. Please check your network connection and the server URL.
      • Fix: Verify network connectivity and that the S2V server URL (if overridden) is correct.

This guide should help you decipher most validation messages. Always pay close attention to the file name, object name, and specific property mentioned in the error or warning to quickly locate and fix the issue in your YAML files.