SYSTEM MACHINE TRASLATION EVALUATION


Abstract
Introduction
Colloquial expressions
Other expressions
Ortographics problems
Words with different meanings
Words in Spanish that are disappearing
Idioms in English
Idioms in Spanish
Phrasal Verbs
Technical translation
Synonyms
Other kind of sentences
Conclusion

ABSTRACT

This report analyses the behaviour of machine translation when translating expressions. These technologies can sometimes help us translate quickly. Nevertheless, the systems often lead to mistaken translations. In this report, some possible causes of the problem are regarded.
First,colloquial expressions used in our daily life to greet or when we meet someone. The next point is another test of these kind of expressions. The third section shows the importance of writing a word ortographically correct. It is impossible for the machine to translate words with such mistakes correctly. The fourth part covers the response of translators to sentences containing words with several possible meanings. Next, words not very commonly used nowadays are tried in the translator. In the sixth section, the expressions typed are idioms in Spanish. They are translated literally, consequently, the obtained sentences are completely wrong. Next, a technical translation is tested. Finally, another expressions have been included. Three different translators have been used for these sentences (Systran, Reverso and Freetranslation).

INTRODUCTION

The topic of this report is “Machine Translation”. Using machine translation makes easier for everybody the search of information. The person that deserves the search will be able to complete an information request in his or her native language and receive the information in that same language, regardless of the language in which the information is recorded. But they have a disadvantage: sometimes natural language is so complex that machine traslation becomes impossible. In the report there are written some examples of lexical ambiguity, structural ambiguity, lexical and structural mismatches, collocations.
The report analyses that translation is not an easy task, above all in a multilingual environment, where the linguistic and computational complexities of machine translation are not always apparent to all users or potential purchasers of systems.
The report is mainly based on proofs done with two machine translators, Systran and Reverso. It is divided in short sections. Their content is elaborated with comparative tables that contain expressions, idioms or technical words written in the space provided for that purpose in the translator, and the results of the translations given by Systran and Reverso machines. The last part is a comparative of the translation of three different systems, Systran, Reverso and Freetranslation.

The first and second chapters contain tables which compare the results of the translations of colloquial and normal expressions elaborated by the machine translators, and finally in the last column we have written what should be the correct translation.
The next point shows how a word written with a slight ortographic mistake is translated by the machine translation in a wrong way.
The reader after reading the fourth chapter will be able to realize that words in English can be translated with different meanings in Spanish. Subsequently, an original chapter comes, where we have learnt the meaning of unusual words in Spanish nowadays and prove wether machine translators are able to traslate them or not.
In some of the last chapters we have observed that translation of idioms and technical words is not an easy work for machine translators.

COLLOQUIAL EXPRESSIONS

SOURCE TARGET 1-SYSTRAN TARGET 2-REVERSO DICTIONARY
Me tomas el pelo You take the hair me You take the hair You are pulling my leg
Me estás tomando el pelo You are taking me the hair You are taking the hair You are pulling my leg
Que te den dos duros That they give two duros you That give you two five-peseta coins
Eres su ojito derecho You are its right ojito You are his(her, your) straight ojito You are the apple of his eye
Voy a comprar un brazo de gitano I am going to buy a gypsy arm I am going to buy a gypsy's arm  
¡Que te vaya bien! Have a good day! That you is OK! Hava a good day!
¿Qué tal estás? What so you are? Such what you are? How are you?
¿Cuantos años tienes? Whichever years you have? All the years you have? How old are you?
How are you ? ¿Cómo es usted? ¿Cómo está Ud.? ¿Qué tal estás?
What is it like? ¿ Cómo cual es? ¿Qué es esto gusta? ¿Cómo es?
How much is it? ¿Cuanto es él? ¿Cuánto es ello? ¿Cuanto es?
¡Hasta pronto! Until soon! Up to(even) soon! See you soon!
See you soon! Véale pronto! Le mirar pronto! Hasta pronto!

OTHER EXPRESSIONS

In this table we can see different expressions translated from English to Spanish and from Spanish to English with two different machine translators.

SOURCE TARGET 1-SYSTRAN TARGET 2-REVERSO DICTIONARY
Creo que sí I believe that if I believe(create) that yes I suppose so
Yo lo cojo I take it I take(catch) it I´ll get /take it
Es una látima It is a pity It(he,she) is a shame It´s a shame
Sano y salvo Safe and sound I recover and save Save and well
A close shave Un afeitado cercano Un afeitado cercano Un afeitado aparado
As far as I am concerned Por lo que me refiero En lo que estoy preocupado Hasta donde yo se
I can´t be bothered No puedo ser incomodado No puedo ser molestado No puedo ser molestado
Anyway De todas formas De todos modos De cualquier manera

ORTOGRAPHIC PROBLEMS

In this table we can see different expressions translated from English to Spanish and from Spanish to English with two different machine translators. Reading them we realize the importance of a good spelling, the translation systems tested translate word by word, they don't understand the context.

In this table we can see different expressions in which depending if we write with tilde or not, the different machines translate them in different ways.

SOURCE TARGET 1-SYSTRAN TARGET 2-REVERSO DICTIONARY
¿Qué hora es? What time is it? What hour(o'clock) is? What time is it?
¿Que hora es? That time is? That hour(o'clock) is? ---
Estás cansado You are tired You are tired(tiring) You are tired
Estas cansado These tired These tired ---
¿Cuánto cuesta? How much it costs? Does how much cost? How much does it cost?
¿Cuanto cuesta? Whichever hill? All that costs? ---

WORDS WITH DIFFERENT MEANINGS

This section includes different sentences with words with several possible translations. What we tried to see is the influence of the context of the sentence in the translation.

SOURCE TARGET 1-SYSTRAN TARGET 2-REVERSO DICTIONARY
I do not know you very well No le conozco muy bien No le conozco muy bien No le(te) conozco muy bien
I know English and French Sé inglés y francés Sé inglés y francés Sé inglés y francés
I can swim Sé nadar Puedo nadar Sé nadar *
I can open the door Puedo abrir la puerta Puedo abrir la puerta Puedo abrir la puerta
I can seldom swim Sé nadar raramente Raras veces puedo nadar Raras veces puedo nadar
I can swim today Sé nadar hoy Puedo nadar hoy Hoy puedo nadar
Puedo nadar hoy I can swim today I can swim today I can swim today
Le he conocido hoy I have known him today I have known him today I met him today
I have met him today Lo he satisfecho hoy Lo he encontrado hoy Le he conocido hoy
Pleased to meet you Satisfecho para satisfacerle Contento encontrarle Encantado de conocerte

*: The translation of "I can swim" could be "puedo nadar", depending on the context, but we expect the translation without any context should be "sé nadar".

We can state that these machine translators work in a similar way for the previous expressions. Systran translates better sentences containing the verb 'can', because it works with more than one meaning in Spanish. It's remarkable that neither Systran nor Reverso recognize the last expression, though it is very common.

WORDS IN SPANISH THAT ARE DISAPPEARING

In this point we have analysed some words that in Spanish aren´t very usual. First, we have explained their meaning. Afterwards, we have done small proofs with the machine translators to know if they know translating them from Spanish to English, sometimes with a sentence.
The first sentence is referred to Systran machine translation and the second to Reverso machine translation


(Palabras en castellano casi en desuso)
Andas:Tablero con dos maderos horizontales que sirve para llevar las imágenes en procesión

Acarrear:Levar los haces desde la tierra a la era con el carro

Acedar:: Poner una cosa ácida

Andorga::Barriga, tripa
Arrabal:Barrio situado en las afueras de una población:

Arrecido: Muerto de frío
Función:Fiesta Mayor del pueblo
Relámpago:Apero de tres dientes curvo para volver la basura
Seta:Persona sosa o tonta
Sisa:Impuesto que tenían que pagar los diferentes productos
Zorrera:Humareda

Others words:
Bádil: Recogedor de metal
Barruntar: Prever e imaginarse alguna cosa que al final es cierta.
Bonete:Especie de gorros de pico
Calsesa: Recogedor de metal
Calesa:Especie de coche de dos ruedas
Cantón: Piedra donde se pone el número de km de una carretera
Chiflar:Silbar
Chisquero:Mechero
Ejido: Terreno comunal de varios pueblos colindantes.
Endino:Mala persona
Estofar:Pintar sobre el oro bruñido
Fato:Tonto
Haragán:Vago
Jumento:Asno,burro
Lebrel:Persona muy espabilada
Poyo:Banco de piedra adosado a las paredes de la casa
Rebojo:Trozo de pan duro
Renchir:Repetir la comida
Sariana:Chaqueta
Tabarra:Dar la lata
Tizón:Brasa apagada y negra
Vernizo:Llover serenito y poca cantidad
Yugo:Instrumento de madera que une para que caminen a la par una parejade animales

In this table we can read the traslation of the machines form Spanish to English of these words:

Word Systran Machine translator Reverso transductor Dictionary
Bádil Bádil Bádil It doesn´t appear
Barruntar Barruntar To feel To guess, to suspect
Bonete Cap Bonnet It doesn´t appear
Calsesa Calsesa Calsesa It doesn´t appear
Calesa Caleche Maggot It doesn´t appear
Cantón Corner Corner It doesn´t appear
Chiflar to Chiflar Chiflar To hiss, to boo
Chisquero Chisquero Chisquero It doesn´t appear
Ejido Ejido Common land, It doesn´t appear
Endino Endino Endino It doesn´t appear
Estofar To estofar To stew To stew, to quilt
Fato Fato Fato It doesn´t appear
Haragán Haragan Idler Good-for-nothing
Jumento Jumento Donkey It doesn´t appear
Lebrel Lebrel Greyhound Greyhound
Poyo Poyo Bench It doesn´t appear
Rebojo Rebojo Rebojo It doesn´t appear

IDIOMS IN ENGLISH

In this table we can see different idioms translated from English to Spanish with two different machine translators.

SOURCE TARGET 1-SYSTRAN TARGET 2-REVERSO Really
To fell under the weather A cayó bajo tiempo A cayeron debajo el tiempo To be depressed
I´m still in a complete fog Todavía estoy en una niebla completa Estoy todavía en una niebla completa I´m confused
To go like the wind Para ir como el viento Ir como el viento Go very fast
Come rain or shine Viene la lluvia o el brillo Venga la lluvia o el brillo Whatever happens
There is a cloud in the horizon Hay una nube en el horizonte Hay una nube en el horizonte Something threatening

The conclusion we can extract from this table is the fact that both systems are unable to translate properly most of these expressions.

IDIOMS IN SPANISH

In this table we can see different idioms translated from Spanish to English with two different machine translations.

SOURCE TARGET 1-SYSTRAN TARGET 2-REVERSO
Poner toda la carne en el asador To go the whole hog To put all the meat in the spit
Cuando marzo mayea, mayo marcea When mayea March, marcea May When March mayea, May marcea
No por mucho madrugar amanece más temprano Not by madrugar amanece much more early Not for much to get up early dawns more early
A quien madruga, Dios le ayuda To who madruga, God him aid Whom it(he,she) forestalls, God helps him(her)
Quien a buen árbol se arrima, buena sombra le cobija Who to good tree brings closer itself, good shade him blanket The one who to good tree comes closer, good shade shelters him
En boca cerrada no entran moscas Closed mouth flies do not enter Closed mouth flies do not enter
Por la boca muere el pez By the mouth the fish dies For the mouth the fish dies
A caballo regalado no le mires el diente At given horse you do not watch the tooth to him Astride given you do not look at the tooth
Cuanto más grande más tonto The more great the more idiot The bigger sillier
En casa del herrero cuchillo de palo In house of the blacksmith wood knife In house of the blacksmith knife of stick(wood)
A buenas horas mangas verdes To good hours green sleeves At good hours sleeves green
De tal palo tal astilla Of such wood so it chips Of such a stick(wood) such a splinter
A ojo de buen cubero By rough estimate of good cubero By the lump of good cubero

PHRASAL VERBS

SOURCE TARGET 1-SYSTRAN TARGET 2-REVERSO TARGET 3-FREETRANSLATION DICTIONARY
Bring forth Produzca Traiga En adelante Dé a luz Dar a luz
Soar up Elévese para arriba Subida encima Elévese arriba Dar coba a alguien
Be fired out Enciéndase hacia fuera Ser encendido(despedido) hacia fuera Sea despedido fuera Ser Despedido
Be a turn on Sea una vuelta encendido Esté una vuelta sobre Sea un prende Ser una tia buena

TECHNICAL TRANSLATION

In this part of the work we have taken a specific test to make proves traslating it from English to Spanish with two different machine translators. The original text is:

In a cable network, data form the network to the user is referred to as downstream, whereas data from the user to the network is referred to as upstream

In the translations writing by the two machines we can observe that that the technical words "upstream" and "downstream" have been translated literally

SYNONYMS

SOURCE TARGET 1-SYSTRAN TARGET 2-REVERSO TARGET 3-FREETRANSLATION DICTIONARY
Face the music Haga frente a la músic Ate la musica Encare la música Afrontar los problemas
Tackle the problem Aborde el problema Aborde el problema Agarre el problema Afrontar problemas

Here we have another couple of synonyms. And we are going to use these kind of gaps, to settle the difference between couples.

SOURCE TARGET 1-SYSTRAN TARGET 2-REVERSO TARGET 3-FREETRANSLATION DICTIONARY
Take an exam Tome un examen Tome un examen Tome un examen Afrontar los problemas
Sit an exam Siente un examen Siente un examen Siéntese un examen Afrontar los problemas

SOURCE TARGET 1-SYSTRAN TARGET 2-REVERSO TARGET 3-FREETRANSLATION DICTIONARY
Go on spree Vaya en juerga Continúes la juerga Pase juerga Salir de juerga
Paint the town red Pinte la ciudad roja Váyase de juerga Pinte el pueblo rojo Salir de juerga

SOURCE TARGET 1-SYSTRAN TARGET 2-REVERSO TARGET 3-FREETRANSLATION DICTIONARY
Set the mind at rest Fije la mente en el resto Ponga la mente descansando Ponga la mente parada Dar un descanso
Set the mind at peace Fije la mente en la paz Ponga la mente en la paz Ponga la mente en la paz Dar un descanso

SOURCE TARGET 1-SYSTRAN TARGET 2-REVERSO TARGET 3-FREETRANSLATION DICTIONARY
Get the boat Consiga el barco Consiga el barco Obtenga el barco Ser despedido
Get the sack Consiga el saco Consiga el saco Obtenga el saco Ser despedido
Make redundant Haga redundante Despedir Haga superfluo Ser despedido

We realize that the translation systems don't recognize synonyms in the cases exposed in the table above.

OTHER KIND OF SENTENCES

Here, we have a mixture of sentences of different types. We have created a different section for these kind of sentences, because, in this part, we wanted to show the differences in another translator system, so as to stand that these machines are not very reliable.

SOURCE TARGET 1-SYSTRAN TARGET 2-REVERSO TARGET 3-FREETRANSLATION DICTIONARY
Phony Jewelry Joyería De Phony Joyería Falsa Las Joyas falsas Bisuteria, baratijas
Gain the top of the mountain Gane la tapa de la montaña Gane la cima de la montaña Gane la cima de la montaña Llegar a la cima
Grudge against Resentimiento contra Rencor contra Envidie contra Rencor
Beat about the bush Golpe sobre el arbusto Golpeado sobre el arbusto Golpee acerca del arbusto Dar rodeos
Rack a nut Agriete una tuerca Raje una tuerca Agriete una nuez
It gives me the creeps Me da se arrastra Esto me da el se arrastra Me da el arrastra Me horroriza
Get ashore Consiga en tierra Póngase en tierra Obtenga a tierra Llegar a la costa
Drift Deriva Flujo El deriva Ir a la deriva
Turn Blank Dé vuelta Al Espacio en blanco Vuelta En blanco Gire Blanco Quedarse en blanco
Pay cash Efectivo de la paga Pagar en efectivo Pague al contado Pagar en metalico
Dark horse Caballo oscuro Caballo oscuro El caballo oscuro Tipo extraño, perro verde
Get cold feet Consiga los pies fríos Haga frío pies Obtenga pies fríos Be afraid
Live from hand to mouth Viva de la mano para articular Vivir al día Viva de al día Vivir al dia
To be all fingers Para ser todos los dedos Ser todos los dedos Para ser todo toca Ser un manazas
To be up to the ears Para estar hasta los oídos Aumentar a los oídos Para estar hasta las orejas Estar saturado de trabajo
To be wet behind the ears Para ser mojado detrás de los oídos Ser mojado detrás de los oídos Para ser mojado atrás las orejas Inexperto
At sixes and sevens En los sixes y los sevens En sixes y sevens En seises y sietes Desorden
On all fours En todos los fours A gatas A gatas a gatas
One over the eight Un excedente los ocho Un sobre los ocho Uno sobre el ocho Borracho
Throw a party for somebody Lance un partido para alguien Dé una fiesta para alguien Tire un partido para alguien Hacer una fiesta a alguien
To trap the fingers Para atrapar los dedo Atrapar los dedos Para atrapar los dedos Pillarse los dedos
Put it in a nutshell Puesto le en una cáscara de nuez Póngalo en una cáscara de nuez Póngalo en una cáscara de nuez Resumir
What a chick Un qué polluelo Qué polluelo Qué un polluelo ¡que morro!
Have the nerve to do it Tenga el nervio para hacerlo Tenga el nervio para hacerlo Tenga el nervio para hacerlo tener el morro de hacerlo
Keep your hair on Guarde su pelo encendido Guarde(Mantenga) su pelo sobre Mantenga el cabello en Tranquilizate
Put you in a picture Puesto le en un cuadro Póngale en un cuadro(una imagen) Póngalo en un retrato Poner al corriente
Turn the pimples Dé vuelta a las espinillas Gire las espinillas Gire los granos Poner la piel de gallina
Take the plunge Tome la zambullida Arriésguese Tome el hunde Dar el paso decisivo
To be on mourning Para estar en el luto Estar sobre luto Para estar en el luto Estar de luto
Wet blanket Manta mojada Aguafiestas La frazada mojada Aguafiestas
Couch Potato Patata Del Sofá Telespectador pasivo La Papa de la sofá Telespectador pasivo
Cup of tea Taza de té Taza de té La copa de té Media naranja
Have a eskeleton on his cupboard Tenga un esqueleto en su armario Tenga un skeleton sobre su armario Tenga un eskeleton en su alacena Guardar un as en la manga
Pull the leg Tire de la pierna Tire la pierna Estire la pierna Vacilar
Fall for the trick Caída para el truco Enamórese del truco Caiga para la artimaña Be deceived
Cross the mind Cruce la mente Cruce la mente Cruce la mente Ocurrirse
Get under my feet Consiga bajo mis pies Póngase bajo mis pies Obtenga abajo mis pies Dando la chapa
Slipped the mind Deslizó la mente Resbalado la mente Resbaló la mente Olvidar
It ring a bell Suena una campana Esto toca una campana Suena una campana me suena
Talk throght the hat Throught de la charla el sombrer Conversación throught el sombrero Hable a traves el sombrero speak nonsese
Put 2 and 2 togehter Ponga 2 y 2 juntos Reúna 2 y 2 Ponga 2 y 2 junto atar cabos
Red tape Cinta roja Papeleo El papeleo Burocracia
Red Herring Arenques Rojos Arenque ahumado(Pretexto para desviar la atención) La Pista Falsa Dar un rodeo
Put the thinking cup on Ponga la taza de pensamiento encendido Ponga la taza pensadora Ponga la copa pensadora en Darle vueltas a algo
Foot the bill Pie la cuenta Pie la cuenta(el proyecto de la ley) Pagar la cuenta Pagar la cuenta
Gain ground Gane terreno Tierra(Razón) de beneficio Gane Ganar aceptación
Get wind of Consiga el viento de Consiga el viento Obtenga viento de Enterarse de algo
In full swing En el oscilación completo En oscilación llena En plena actividad En pleno apogeo
At full blast En la ráfaga completa A todo volumen En al máximo A tope
No end Ningún extremo Ningún final Ningún fin Mucho
Turn a blind eye Dé vuelta a un ojo oculto Hacer la vista gorda Hacer la vista gorda
Flying colours Colores de vuelo Colores volantes Con honores
Out of the blue Fuera del azul Como por arte de magia Gire un ojo ciego De repente
Keep body and soul together Cuerpo y alma de la subsistencia junto Guarde(mantenga) el cuerpo y el alma juntos Mantenga el cuerpo y el alma juntos Survive
Call it a day Llámelo un día Llamele un dia Llámele un día Se acabo por hoy
Steal a glance Robe un vistazo Robe un vistazo Hurte una mirada Echar una mriada
Let the cat out of the bag Deje el gato fuera del bolso Deje al gato del bolso Permita el gato fuera de la bolsa Revelar un secreto
Lump of sugar Terrón del azúcar Terrón de azúcar Amontone de azúcar Terron de azucar
Plead guilty Abogue por culpable Declarar culpable Implore culpable Declarar culpable
Have the guts to do it Tenga la tripa para hacerlo Tenga la tripa a hacerlo Tenga la bravura suficiente para hágalo Tener agallas
Play the fools Juegue a tontos Juegue a los tontos Juegue los tontos hacer el tonto
Get into hot water Consiga en la agua caliente entre agua caliente Entre en agua caliente meterse en problemas
Run short of time Funcione brevemente de tiempo Quedado tiempo Corra brevemente de tiempo Con poco tiempo
Play by ear Al juego por el oído Juego por oído Juegue por oreja Improvisar
Be in good spirits Be in good spirits Esté de buen humor Esté en espíritus buenos In the glory
See nothing amiss No vea nada inoportunamente No mirar nada mal No vea nada malo No veo nada extraño
To be on cloud nine Para estar en la nube nueve Estar sobre nube nueve Para estar en la nube nueve En el 7º cielo
A sack and bull story Una historia del saco y del toro Un saco e historia de toro
Big headed Grande dirigido Grande encabezado Grande dirigió Engreido
In the gut En la tripa En la tripa En el intestino Actuar al fnal
Beaten track Pista batida Pista golpeada El vestigio golpeado Novedoso

CONCLUSION

We haven´t forgotten that traslating is a difficult task , and if a translation is done by a machine some problems can appear like lexical ambiguity, collocations ...etc. In the report, we have observed the following weak points of machine traslators:
1. Most of the times, the context in which a word is written is not recognized and sentences are translated literally
2. Words that the system doesn't understand are not translated
However, machine translations sometimes are clever:
a) If a word is unusual but they know it they look for a synonym and traslate it.
b) If they understand the meaning of the words they distinguish them morfologically, that is, they classify words as verbs, nouns, adjectives...

As far as we are concerned, in the creation of a reliable machine translation, there are two conditions to follow:
1. It is necessary to balance the participation between the human and machine translation, that is, a good interactivity between user and machine should be developed.
2. The machine must be able to help the person who is traslating, it has to know examples, terms models..., etc