Difference between revisions of "Automatic Feed Translation"

From AwasuWiki
Jump to: navigation, search
(Moved Google Translate API deprecation notice to top of page)
 
(9 intermediate revisions by 3 users not shown)
Line 1: Line 1:
 +
<pre style="background-color:pink;color:firebrick;font-weight:bold;">
 +
Google has announced that the Google Translate API, upon which this Awasu extension is based, has been officially
 +
deprecated as of May 26, 2011. "...the number of requests you may make per day will be limited and the API will be shut
 +
off completely on December 1, 2011."
 +
 +
http://code.google.com/apis/language/translate/overview.html
 +
 +
We're in the process of seeking alternative translation functionality to provide Awasu users with the same
 +
or similar translation functionality.
 +
</pre>
 +
 
=Automatic Feed Translation=
 
=Automatic Feed Translation=
  
Using the XSLT file listed below you can automatically translate a foreign language feed into your language, utilizing [http://code.google.com/apis/ajaxlanguage/ Google's Translation API], whenever Awasu updates the associated Channel.  Here is the list of [http://code.google.com/apis/ajaxlanguage/documentation/#SupportedLanguages supported languages].
+
You can automatically translate a foreign language feed into your language, utilizing [http://code.google.com/apis/ajaxlanguage/ Google's Translation API], whenever Awasu updates the associated Channel.  Here is the list of [http://code.google.com/apis/ajaxlanguage/documentation/#SupportedLanguages supported languages].
 +
 
 +
Download and save this file somewhere:
 +
    [http://www.awasu.com/downloads/extensions/AutoTranslate/AutoTranslate-v2.xsl AutoTranslate-v2.xsl]
 +
 
 +
<small><em>NOTE: Awasu was upgraded in v2.4.3 to use MSXML6 which contains changes that break this XSL. A fix has already been made and will be released in 2.4.4.alpha1.</em></small>
  
 
==Customising the XSLT file==
 
==Customising the XSLT file==
Line 9: Line 25:
  
 
===Feed language===
 
===Feed language===
There is a constant in the XSLT file called "feedLanguage" which you can change to a specific [http://code.google.com/apis/ajaxlanguage/documentation/reference.html#LangNameArray two-character language code] if you know the language of the feed; this will improve Google's language translation accuracy.  Leaving the "feedLanguage" constant blank (''empty quotation marks: ""'') should still work as Google will attempt to guess the feed's langauge.
+
There is a constant in the XSLT file called "feedLanguage" which you can change to a specific [http://code.google.com/apis/ajaxlanguage/documentation/reference.html#LangNameArray two-character language code] if you know the language of the feed; this will improve Google's language translation accuracy.  Leaving the "feedLanguage" constant blank (''empty quotation marks: ""'') should still work as Google will attempt to guess the feed's language. More information about setting the feed language is provided below under the "'''Error Reporting'''" section.
 
 
<pre>
 
<?xml version="1.0" encoding="UTF-8"?>
 
<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
 
                              xmlns:msxsl="urn:schemas-microsoft-com:xslt"
 
                              xmlns:ktc="http://www.awasu.com/forums/profile.php?mode=viewprofile&amp;u=24618"
 
                              xmlns:atom="http://www.w3.org/2005/Atom"
 
                              xmlns:content="http://purl.org/rss/1.0/modules/content/">
 
  <xsl:output method="xml" version="1.0" encoding="UTF-8" indent="yes"/>
 
  <msxsl:script language="JScript" implements-prefix="ktc"><![CDATA[
 
  /*
 
  Name: AutoTranslate.xsl
 
 
 
  Description: Automatically translates Atom and RSS feed items received in Awasu to your language using Google's translation service.
 
 
 
  Author: kevotheclone (http://www.awasu.com/forums/profile.php?mode=viewprofile&amp;u=24618)
 
 
 
  For additional documentation: http://www.awasu.com/wiki/Automatic_Feed_Translation
 
  */
 
 
 
  // Constants...
 
 
 
  // Don't edit these two constants until Google says otherwise.
 
  var baseURL = "http://ajax.googleapis.com/ajax/services/language/translate";
 
  var version = "v=1.0";
 
 
 
  /*
 
  Change this constant (feedLanguage) to the two-character language code of the feed (if known).
 
  This will improve Google language translation accuracy.
 
 
 
  Leave is blank (empty quotation marks: "") if the feed language is unknown or
 
  the feed contains multiple languages. Google will attempt to guess the langauge
 
  each time the translateLang() function is called.
 
 
 
  Supported language codes are listed here:
 
  http://code.google.com/apis/ajaxlanguage/documentation/reference.html#LangNameArray
 
  */
 
  var feedLanguage = "";
 
 
 
  /*
 
  Change this constant (yourLanguage) to your desired two-character language code.
 
  Supported language codes are listed here:
 
  http://code.google.com/apis/ajaxlanguage/documentation/reference.html#LangNameArray
 
  */
 
  var yourLanguage = "en";
 
 
 
  var xmlhttp = new ActiveXObject("Msxml2.XMLHTTP.4.0");
 
  var cache = {}; // Case-sensitive results cache.
 
  var chunks;    // Array of Strings, for breaking up large text into multiple HTTP calls.
 
 
 
  var WshShell = new ActiveXObject("WScript.Shell"); // For logging error messages.
 
 
 
  function translateText(textToTranslate)
 
  {
 
    var translatedText = "";
 
    try
 
    {
 
      textToTranslate = textToTranslate.replace(/^\s+|\s+$/g,"");            // Remove leading and trailing whitespace.
 
      textToTranslate = textToTranslate.replace(/^\s*|\s(?=\s)|\s*$/g," "); // Replace repeated spaces, newlines and tabs with a single space.
 
  
      if (cache[textToTranslate])        // It's it in the cache,
+
==Error Reporting==
        return cache[textToTranslate]; // return it.
+
Due to the nature of XSLT processors, it is impossible to output error information in such a way that Awasu could capture it and display it in the Channel Properties dialog box.  So as an alternative error reporting mechanism, any errors will be logged in the Windows "Application" Event Log.  Two types of errors are reported: 1) Problems connecting to the Google Translation service, and 2) problems reported by the the Google Translation service.
  
      if (textToTranslate)
+
These events will have "'''WSH'''" as the "'''Source'''":
      {
+
[[image:AutoTranslateEventViewer.jpg]]
  
        chunks = textToTranslate.match(/.{2000,4000}\b|.+$/g);
+
Double-clicking on one of the events will show the detail error code and description of the problem:
 +
[[image:AutoTranslateEventProperties.jpg]]
  
        for (var i=0; i<chunks.length; i++)
+
The most common problem I encountered when testing this XSLT was a "'''400'''" error code from Google with this description "'''could not reliably detect source language'''".  By setting the feed language to a specific [http://code.google.com/apis/ajaxlanguage/documentation/reference.html#LangNameArray two-character language code] I was able to eliminate this type of error completely. If a feed item fails to translate you can manually translate it using these [[http://www.awasu.com/wiki/Google_User_Tools#Language_translation_Send_to_tool Send to/User tools]].
        {
 
          xmlhttp.open("POST", baseURL, false);
 
          xmlhttp.setRequestHeader("Referer", "http://www.awasu.com/forums/profile.php?mode=viewprofile&amp;u=24618");
 
          xmlhttp.setRequestHeader("Content-Type", "application/x-www-form-urlencoded");
 
          xmlhttp.send("v=1.0&q=" + encodeURIComponent(chunks[i]) + "&langpair=" + feedLanguage + "|" + yourLanguage);
 
          eval("var response = " + xmlhttp.responseText);
 
          /*
 
          If successful, concatenate the translated chunk of text to our output buffer,
 
          else, if any chunk fails return the original untranslated text.
 
          */
 
          if (xmlhttp.status == 200)
 
          {
 
            if (response.responseStatus == 200) // check responseStatus, 200=success, 400=error
 
              translatedText += encodeURI(response.responseData.translatedText);
 
            else
 
            {
 
              WshShell.LogEvent(1, "Error from Google's Translate service.\r\n\r\nStatus code and description:\r\n" +
 
                                  response.responseStatus + " - " + response.responseDetails + "\r\n\r\nText to translate:\r\n" + textToTranslate);
 
              return textToTranslate;
 
            }
 
          }
 
          else
 
          {
 
            WshShell.LogEvent(1, "Error connecting to Google's translate service.\r\n\r\nHTTP Status code and description:\r\n" +
 
                                xmlhttp.status + " - " + xmlhttp.statusText);
 
            return textToTranslate;
 
          }
 
        }
 
        cache[textToTranslate] = decodeURI(translatedText); // Add the results to the cache.
 
        return decodeURI(translatedText);
 
      }
 
      else // The element's value is a null string.
 
      {
 
        cache[textToTranslate] = "";
 
        return "";
 
      }
 
    }
 
    catch(e)
 
    {
 
      cache[textToTranslate] = e;
 
      return e;
 
    }
 
  }
 
  
  function itsBetterWithBacon(textToTranslate)
+
So you may want to keep the AutoTranslate.xsl file with the field language blank, but make copies of it and add specific [http://code.google.com/apis/ajaxlanguage/documentation/reference.html#LangNameArray two-character language code] of the languages that you translate the most.  You might rename the copied files something like this: AutoTranslate_ES_to_EN.xsl ('''Spanish to English'''), AutoTranslate_PL_to_EN.xsl ('''Polish to English'''), etc.
  {
 
    return "bacon: " + textToTranslate + " :bacon";
 
  }
 
 
 
]]></msxsl:script>
 
  <xsl:template match="node()|@*">
 
    <xsl:copy>
 
      <xsl:apply-templates select="@*"/>
 
      <xsl:apply-templates/>
 
    </xsl:copy>
 
  </xsl:template>
 
 
 
  <!-- Atom-based template match patterns -->
 
  <xsl:template match="/atom:feed/atom:title | /atom:feed/atom:subtitle | /atom:feed/atom:entry/atom:title | /atom:feed/atom:entry/atom:content | /atom:feed/atom:entry/atom:category/@term">
 
    <xsl:call-template name="translateText"/>
 
  </xsl:template>
 
 
 
  <!-- RSS-based template match patterns -->
 
  <xsl:template match="/rss/channel/title | /rss/channel/description | /rss/channel/item/title | /rss/channel/item/description | /rss/channel/item/content:encoded | /rss/channel/item/category">
 
    <xsl:call-template name="translateText"/>
 
  </xsl:template>
 
 
 
<!-- Named template that calls the embedded JavaScript function that translates the text -->
 
  <xsl:template name="translateText">
 
    <xsl:variable name="elementName" select="name()"/>
 
    <xsl:element name="{name(.)}">
 
      <xsl:value-of select="ktc:translateText(normalize-space(.))"/>
 
    </xsl:element>
 
  </xsl:template>
 
</xsl:stylesheet>
 
</pre>
 

Latest revision as of 20:25, 1 June 2011

Google has announced that the Google Translate API, upon which this Awasu extension is based, has been officially
deprecated as of May 26, 2011. "...the number of requests you may make per day will be limited and the API will be shut
off completely on December 1, 2011."

http://code.google.com/apis/language/translate/overview.html

We're in the process of seeking alternative translation functionality to provide Awasu users with the same
or similar translation functionality.

Automatic Feed Translation

You can automatically translate a foreign language feed into your language, utilizing Google's Translation API, whenever Awasu updates the associated Channel. Here is the list of supported languages.

Download and save this file somewhere:

   AutoTranslate-v2.xsl

NOTE: Awasu was upgraded in v2.4.3 to use MSXML6 which contains changes that break this XSL. A fix has already been made and will be released in 2.4.4.alpha1.

Customising the XSLT file

Your language

There is a constant in the XSLT file called "yourLanguage" which you can change to a specific two-character language code of your language. If your language is English, the XSLT file below should work well without any modification.

Feed language

There is a constant in the XSLT file called "feedLanguage" which you can change to a specific two-character language code if you know the language of the feed; this will improve Google's language translation accuracy. Leaving the "feedLanguage" constant blank (empty quotation marks: "") should still work as Google will attempt to guess the feed's language. More information about setting the feed language is provided below under the "Error Reporting" section.

Error Reporting

Due to the nature of XSLT processors, it is impossible to output error information in such a way that Awasu could capture it and display it in the Channel Properties dialog box. So as an alternative error reporting mechanism, any errors will be logged in the Windows "Application" Event Log. Two types of errors are reported: 1) Problems connecting to the Google Translation service, and 2) problems reported by the the Google Translation service.

These events will have "WSH" as the "Source": AutoTranslateEventViewer.jpg

Double-clicking on one of the events will show the detail error code and description of the problem: AutoTranslateEventProperties.jpg

The most common problem I encountered when testing this XSLT was a "400" error code from Google with this description "could not reliably detect source language". By setting the feed language to a specific two-character language code I was able to eliminate this type of error completely. If a feed item fails to translate you can manually translate it using these [Send to/User tools].

So you may want to keep the AutoTranslate.xsl file with the field language blank, but make copies of it and add specific two-character language code of the languages that you translate the most. You might rename the copied files something like this: AutoTranslate_ES_to_EN.xsl (Spanish to English), AutoTranslate_PL_to_EN.xsl (Polish to English), etc.