test-model-14

This is a merge of pre-trained language models created using mergekit.

Merge Details

Merge Method

This model was merged using the SCE merge method using /workspace/electra2 as a base.

Models Merged

The following models were included in the merge:

/workspace/cache/models--bruhzair--ignore-merge-6/snapshots/87658005d40b593ba3e87e92e5fb3f28321266a1
/workspace/cache/models--bruhzair--ignore-merge-15/snapshots/e60ee5422f827ab08ac0f591cf18fb7f43f629e8
/workspace/cache/models--bruhzair--ignore-merge-7/snapshots/e3146f6352c469e3798634e02be758a96d1c5b2c
/workspace/cache/models--bruhzair--ignore-merge-21/snapshots/88c4df5f3c979ab849c36fb99b11f15330ac2ce7

Configuration

The following YAML configuration was used to produce this model:

models:
  - model: /workspace/cache/models--bruhzair--ignore-merge-6/snapshots/87658005d40b593ba3e87e92e5fb3f28321266a1
    parameters:
      select_topk: 0.2
  - model: /workspace/cache/models--bruhzair--ignore-merge-7/snapshots/e3146f6352c469e3798634e02be758a96d1c5b2c
    parameters:
      select_topk: 0.2
  - model: /workspace/electra2
    parameters:
      select_topk: 0.25
  - model: /workspace/cache/models--bruhzair--ignore-merge-15/snapshots/e60ee5422f827ab08ac0f591cf18fb7f43f629e8
    parameters:
      select_topk: 0.4
  - model: /workspace/cache/models--bruhzair--ignore-merge-21/snapshots/88c4df5f3c979ab849c36fb99b11f15330ac2ce7
    parameters:
      select_topk: 0.1
base_model: /workspace/electra2
merge_method: sce
tokenizer:
  source: base
dtype: float32
out_dtype: bfloat16