ng et al.)](https://arxiv.org/abs/2009.13658).
        classifier_dropout (`float`, *optional*):
            The dropout ratio for the classification head.
        enable_pronunciation (`bool`, *optional*, defaults to `True`):
            Whether or not the model use pronunciation embed when training.
        enable_shape (`bool`, *optional*, defaults to `True`):
            Whether or not the model use shape embed when training.
        pronunciation_embed_dim (`int`, *optional*, defaults to 768):
            Dimension of the pronunciation_embed.
        pronunciation_vocab_size (`int`, *optional*, defaults to 910):
            Pronunciation Vocabulary size of the RoCBert model. Defines the number of different tokens that can be
            represented by the `input_pronunciation_ids` passed when calling [`RoCBertModel`].
        shape_embed_dim (`int`, *optional*, defaults to 512):
            Dimension of the shape_embed.
        shape_vocab_size (`int`, *optional*, defaults to 24858):
            Shape Vocabulary size of the RoCBert model. Defines the number of different tokens that can be represented
            by the `input_shape_ids` passed when calling [`RoCBertModel`].
        concat_input (`bool`, *optional*, defaults to `True`):
            Defines the way of merging the shape_embed, pronunciation_embed and word_embed, if the value is true,
            output_embed = torch.cat((word_embed, shape_embed, pronunciation_embed), -1), else output_embed =
            (word_embed + shape_embed + pronunciation_embed) / 3
        Example:

    ```python
    >>> from transformers import RoCBertModel, RoCBertConfig

    >>> # Initializing a RoCBert weiweishi/roc-bert-base-zh style configuration
    >>> configuration = RoCBertConfig()

    >>> # Initializing a model from the weiweishi/roc-bert-base-zh style configuration
    >>> model = RoCBertModel(configuration)

    >>> # Accessing the model configuration
    >>> configuration = model.config
    ```Zroc_berté:w