Indonesian to English Translation

Model by

This model translates text from Indonesian to English. It accepts UTF-8 Indonesian text as input and outputs a translated version of the same text in English. This model can be used to translate text from Indonesian to English.

  • Description

    Product Description

    PERFORMANCE METRICS

    This model achieves a Bleu score of 40.82%. The Bleu is a commonly used metric for translation tasks and it is a way of comparing model generated text to a gold standard to see how similar they are.

    40.8% Bleu Score – A method for assessing the quality of text that has been machine-translated from one language to another. The closer the machine translation is to expert human translation, the better the score.Further information here.

    OVERVIEW:

    This model uses the Google Transformer architecture, which is currently the basis for many state-of-the-art translation models. The essence of the Transformer model is the encode-decoder architecture with Attention. Multiple encoders are stacked on top of each other with each one consisting of a self-attention layer, which tries to take into account the full sentence when translating instead of just the word that it is looking at, and a feed-forward neural network. Word embeddings are fed through these encoding layers and are then passed into the decoding layers. In the decoding layer, the self-attention layer only pays attention to earlier positions as opposed to the encode, which allows both directions. Otherwise they work the same way except that the decoding layer attempts to decode the input into the output language. All units have an add and a normalize layer as well. Additionally, in the standard transformer model, there are eight attention heads which are used. The results of these are concatenated into the feed-forward network and then reduced to the correct size. Positional encoding is also used to account for the order of words in the input sequence. This model was trained for 200,000 steps on 4 gpus. The transformer used by this model is described here. The implementation was created using the open source opennmt framework available at opennmt.net.

    TRAINING:

    This model is trained on the Global Voices, Gnome, Infopankki, JW300, KDE4, Opensubtitles, QED, Tanzil, Ted, Tep, Ubuntu and Wikipedia corpora. These total 10,765,836 lines of parallel text. These texts can be found at opus.nlpl.eu.

    VALIDATION:

    This model was validated on 2,500 parallel sentences and achieves a Bleu Score of 40.82%.

    INPUT SPECIFICATION

    The input(s) to this model must adhere to the following specifications:

    Filename Maximum Size Accepted Format(s)
    input.txt 1M .txt

    The “input.txt” file should contain utf-8 encoded Indonesian text.

    OUTPUT DETAILS

    This model will output the following:

    Filename Maximum Size Format
    results.json 1M .json

    The “results.json” file will contain the translated text in the following format: {"text": "text translated from Indonesian"}