Automatic evaluation metrics are essential to the development of machinetranslation systems, but most metrics’ reliance on reference translations limitstheir ability to accept other valid translations.
In this work, we investigate deep, semantic-based approaches to evaluatingmachine translation by directly comparing source and target texts, without usingreferences. We start by extending existing sentence similarity architectures to thecross-lingual case, investigate different training schemes and datasets to deal withthe lack of suitable training data, and finally propose an alignment-based archi-tecture for measuring the meaning overlap in two sentences in an interpretableway. Though our results are not competitive with modern metrics, we demon-strate that useful information for translation evaluation can be learned from othertasks.