Open In Colab

project-logo Train voice styles(.json files) for Supertone/supertonic-3 model.

✨ Key Features

  • Low GPU footprint: peak memory ~2.6 GB.
  • Fast training: typical run time ranges from ~6 to 30 minutes depending on the source voice and dataset size.
  • Flexible: works for single-speaker cloning and style extraction; training to capture accents or complex prosody may take longer.

🎧 Generated Samples

Audio Description Listen Link
Source (F6) ▶️ Play Audio
Styled generation(F6) ▶️ Play Audio
F7 (en -> hi) ▶️ Play Audio

▶️ Script Usage

Example training run with the most commonly changed arguments:

python train_style.py \
  --name my-voice \
  --target-wav-path voices/my-voice.wav \
  --num-steps 3000 \
  --learning-rate 0.0002

Important arguments:

  • --name: output folder/name for logs and the saved style JSON.
  • --target-wav-path: path to the source WAV used for style extraction.
  • --reference-style: use auto to pick the closest built-in style, none for random initialization, or a path to a specific style checkpoint JSON file.
  • --seed, --speed, --num-steps, --learning-rate, --vocoder-steps: core training knobs that affect stability and convergence.

Limitations

While this approach captures speaker identity, its ability to clone emotional expressiveness is limited. This is because the current loss function prioritizes identity over prosody. Furthermore, the original speaker encoder was not released by the developers. Check the samples/ folder for examples of the current results.

Responsible Use & Ethical Considerations

This project utilizes voice cloning technology (Supertone & SpeechBrain). By using this software, you agree to abide by the following ethical guidelines and the Open RAIL-M license restrictions:

  1. Consent: You must obtain explicit permission from the original speaker before cloning their voice.
  2. Deception: Do not use cloned voices to deceive, defraud, or impersonate individuals for malicious purposes (e.g., social engineering, spreading misinformation).
  3. Disclosure: If you share audio generated by this tool publicly, it is highly recommended to add a disclaimer stating that the audio is AI-generated.
  4. Illegal Acts: You may not use this tool to violate any local or international laws, including those regarding harassment, hate speech, or defamation.
  5. Harm to Minors: Usage of this model for the exploitation or harm of minors is strictly prohibited.

Non-Endorsement: The authors of this repository do not condone the use of this technology for creating "deepfakes" or non-consensual content.

Credits & Attribution

@misc{kim2026supertonicembed,
  author       = {Gyeongmin Kim},
  title        = {Extracting Voice Styles from Frozen TTS Models via Gradient-Based Inverse Optimization},
  year         = {2026},
  publisher    = {Zenodo},
  doi          = {10.5281/zenodo.19646514},
  url          = {https://doi.org/10.5281/zenodo.19646514}
}

Models Used

  • supertonic-3: Licensed under Open RAIL-M. Note: This license includes "Responsible AI" usage restrictions.
  • spkrec-ecapa-voxceleb: The spkrec-ecapa-voxceleb model is used under the Apache License 2.0.