We could not stand the temptation to implement the @Yield annotation that we described earlier.

Idea is rather clear but people are saying that it's not an easy task to update the sources.

They were right!

Implementation has its price, as we were forced to access JDK's classes of javac compiler. As result, at present, we don't support other compilers such as EclipseCompiler. We shall look later what can be done in this area.

At present, annotation processor works perfectly when you run javac either from the command line, from ant, or from other build tool.

Here is an example of how method is refactored:

@Yield
public static Iterable<Long> fibonachi()
{
  ArrayList<Long> items = new ArrayList<Long>();

  long Ti = 0;
  long Ti1 = 1;

  while(true)
  {
    items.add(Ti);

    long value = Ti + Ti1;

    Ti = Ti1;
    Ti1 = value;
  }
}

And that's how we transform it:

@Yield()
public static Iterable<Long> fibonachi() {
  assert (java.util.ArrayList<Long>)(ArrayList<Long>)null == null : null;

  class $state$ implements java.lang.Iterable<Long>, java.util.Iterator<Long>, java.io.Closeable {

    public java.util.Iterator<Long> iterator() {
      if ($state$id == 0) {
        $state$id = 1;
        return this;
      } else return new $state$();
    }

    public boolean hasNext() {
      if (!$state$nextDefined) {
        $state$hasNext = $state$next();
        $state$nextDefined = true;
      }

      return $state$hasNext;
    }

    public Long next() {
      if (!hasNext()) throw new java.util.NoSuchElementException();

      $state$nextDefined = false;

      return $state$next;
    }

    public void remove() {
      throw new java.lang.UnsupportedOperationException();
    }

    public void close() {
      $state$id = 5;
    }

    private boolean $state$next() {
      while (true) switch ($state$id) {
      case 0:
        $state$id = 1;
      case 1:
        Ti = 0;
        Ti1 = 1;
      case 2:
        if (!true) {
          $state$id = 4;
          break;
        }

        $state$next = Ti;
        $state$id = 3;

        return true;
      case 3:
        value = Ti + Ti1;
        Ti = Ti1;
        Ti1 = value;
        $state$id = 2;

        break;
      case 4:
      case 5:
      default:
        $state$id = 5;

        return false;
      }
    }

    private long Ti;
    private long Ti1;
    private long value;
    private int $state$id;
    private boolean $state$hasNext;
    private boolean $state$nextDefined;
    private Long $state$next;
  }

  return new $state$();
}

Formatting is automatic, sorry, but anyway it's for diagnostics only. You will never see this code.

It's iteresting to say that this implementation is very precisely mimics xslt state machine implementation we have done back in 2008.

You can download YieldProcessor here. We hope that someone will find our solution very interesting.

You might be interested in the following article that was written in form of a little guide. It can educate about new ways to learn SQL and hopefully may help someone to get a job. See "How to get MS SQL certification" that was written by Michele Rouse.

Several times we have already wished to see yield feature in java and all the time came to the same implementation: infomancers-collections. And every time with dissatisfaction turned away, and continued with regular iterators.

Why? Well, in spite of the fact it's the best implementation of the feature we have seen, it's still too heavy, as it's playing with java byte code at run-time.

We never grasped the idea why it's done this way, while there is post-compile time annotation processing in java.

If we would implemented the yeild feature in java we would created a @Yield annotation and would demanded to implement some well defined code pattern like this:

@Yield
Iteratable<String> iterator()
{
  // This is part of pattern.
  ArrayList<String> list = new ArrayList<String>();

  for(int i = 0; i < 10; ++i)
  {
    // list.add() plays the role of yield return.
    list.add(String.valueOf(i));
  }

  // This is part of pattern.
  return list;
}

or

@Yield
Iterator<String> iterator()
{
  // This is part of pattern.
  ArrayList<String> list = new ArrayList<String>();

  for(int i = 0; i < 10; ++i)
  {
    // list.add() plays the role of yield return.
    list.add(String.valueOf(i));
  }

  // This is part of pattern.
  return list.iterator();
}

Note that the code will work correctly even, if by mischance, post-compile-time processing will not take place.

At post comile time we would do all required refactoring to turn these implementations into a state machines thus runtime would not contain any third party components.

It's iteresting to recall that we have also implemented similar refactoring in pure xslt.

See What you can do with jxom.

Update: implementation can be found at Yield.zip

We have a class Beans used to serialize a list of generic objects into an xml. This is done like this:

public class Call
{
  public Beans input;
  public Beans output;
  ...
}

@XmlJavaTypeAdapter(value = BeanAdapter.class)
public class Beans
{
  public List<Object> bean;
}

Thanks to @XmlJavaTypeAdapter, we're able to write xml in whatever form we want.

When we're serializing a Call instance:

Call call = ...
Beans beans = ...;

call.setInput(beans);

JAXBContext context = ...;
Marshaller marshaler = context.createMarshaller();
ObjectFactory factory = ...;

marshaler.marshal(factory.createCall(call), result);

things work as expected, meaning that BeanAdapter is used during xml serialization. But if it's happened that you want to serialize a Beans instance itself, you start getting problems with the serialization of unknown objects. That's because JAXB does not use BeanAdapter.

We have found a similar case "How to assign an adapter to the root element?", unfortunately with no satisfactory explanation.

That is strange.

Last Thursday, 18 Nov 2010, we with our colleagues from BluePhoenix were in a trip at Wadi Kelt. See our photo-report here.

Michael Key, author of the Saxon xslt processor, being inspired by the GWT ideas, has decided to compile Saxon HE into javascript. See Compiling Saxon using GWT.

The resulting script is about 1MB of size.

But what we thought lately, that it's overkill to bring whole xslt engine on a client, while it's possible to generate javascript from xslt the same way as he's building java from xquery. This will probably require some runtime but of much lesser size.

Search at www.google.fr: An empty sequence is not allowed as the @select attribute of xsl:analyze-string

That's known issue. See Bug 7976.

In xslt 2.0 you should either check the value before using xsl:analyze-string, or wrap it into string() call.

The problem is addressed in xslt 3.0

michaelhkay: Saxon 9.3 has been out for 8 days: only two bugs so far, one found by me. I think that's a record.

Not necessary. We, for example, who use Saxon HE, have found nothing new in Saxon 9.3, while expected to see xslt 3.0. Disappointed. No actual reason to migrate.

P.S. We were among the first who were finding early bugs in previous releases.

Reading individual papers of C++ WG, you can find the following one:

There, Bjarne Stroustrup thinks about issues with implicitly generated copy and move operations in C++.

It's always a pleasure to see how one can deal with a problem burdened with antagonisms. To conduct his position Bjarne skilfully uses not only rational but also emotional argumentation:

...We may deem this “bad code that deserves to be broken” or “unrealistic”, but this example demonstrates that the problem with a generated move has an exact counterpart for copy (which we have lived with for 27 years)...

...In 1984, I missed the chance to protect us against copy and we have lived with the problems ever since. I should have instituted some rule along the lines “if a class has a destructor, no copy operations are generated” or “if a class has a pointer member, no copy operations are generated.”...

It's impossible to recall this numbers without shivering. :-)

We're following w3's "Bug 9069 - Function to invoke an XSLT transformation".

There, people argue about xpath API to invoke xslt transformations. Function should look roughly like this:

transform
(
  $node-tree as node()?,
  $stylesheet as item(),
  $parameters as XXX
) as node()

The discussion is spinning around the last argument: $parameters as XXX. Should it be an xml element describing parameters, a function returning values for parameter names, or some new type modelling immutable map?

What is most interesting in this discussion is the leak about plans to introduce a map type:

There is a hope that map will finally appear in xslt!

See also:
Bug 5630 - [DM] Tuples and maps,
Tuples and maps - Status: CLOSED, WONTFIX,
Map, based on immutable trees,
Maps in exslt2?

Historically jxom was developed first, and as such exhibited some imperfectness in its xml schema. csharpxom has taken into an account jxom's problems.

Unfortunately we could not easily fix jxom as a great amount of code already uses it. In this refactoring we tried to be conservative, and have changed only "type" and "import" xml schema elements in java.xsd.

Consider type reference and package import constructs in the old schema:

<!-- import java.util.ArrayList; -->
<import name="java.util.ArrayList"/>

<!-- java.util.ArrayList<java.math.BigDecimal> -->
<type package="java.util">
  <part name="ArrayList">
    <argument>
      <type name="BigDecimal" package="java.math">
    </argument>
  </part>
</type>

<!-- my.Parent.Nested -->
<type package="my">
  <part name="Parent"/>
  <part name="Nested"/>
<type>

Here we can observe that:

• type is referred by a qualified name in import element;
• type has two forms: simple (see BigDecimal), and other for nested or generic type (see ArrayList).

We have made it more consistent in the updated jxom:

<!-- import java.util.ArrayList; -->
<import>
  <type name="ArrayList" package="java.util"/>
</import>

<!-- java.util.ArrayList<java.math.BigDecimal> -->
<type name="ArrayList" package="java.util">
  <argument>
    <type name="BigDecimal" package="java.math">
  </argument>
</type>

<!-- my.Parent.Nested -->
<type name="Nested">
  <type name="Parent" package="my"/>
<type>

We hope that you will not be impacted very much by this fix.

Please refresh Languages XOM from languages-xom.zip.

P.S. we have also included xml schema and xslt api to generate ASPX (see Xslt serializer for ASPX output). We, in fact, in our projects, generate aspx documents with embedded csharpxom, and then pass it through two stage transformation.

In the previous post we have announced an API to parse a COBOL source into the cobolxom.

We exploited the incremental parser to build a grammar xml tree and then were planning to create an xslt transformation to generate cobolxom.

Now, we would like to declare that such xslt is ready.

At present all standard COBOL constructs are supported, but more tests are required. Preprocessor support is still in the todo list.

You may peek into an examples of COBOL:

Cobol grammar:

And cobolxom:

While we were building a grammar to cobolxom stylesheet we asked ourselves whether the COBOL parsing could be done entirely in xslt. The answer is yes, so who knows it might be that we shall turn this task into pure xslt one. :-)

Recently we've seen a code like this:

<xsl:variable name="a" as="element()?" select="..."/>
<xsl:variable name="b" as="element()?" select="..."/>

<xsl:apply-templates select="$a">
  <xsl:with-param name="b" tunnel="yes" as="element()" select="$b"/>
</xsl:apply-templates>

It fails with an error: "An empty sequence is not allowed as the value of parameter $b".

What is interesting is that the value of $a is an empty sequence, so the code could potentially work, provided processor evaluated $a first, and decided not to evaluate xsl:with-param.

Whether the order of evaluation of @select and xsl:with-param is specified by the standard or it's an implementation defined?

We asked this question on xslt forum, and got the following answer:

The specification leaves this implementation-defined. Since the values of the parameters are the same for every node processed, it's a reasonably strategy for the processor to evaluate the parameters before knowing how many selected nodes there are, though I guess an even better strategy would be to do it lazily when the first selected node is found.

Well, that's an expected answer. This question will probably induce Michael Kay to introduce a small optimization into the Saxon.

Once ago we have created an incremental parser, and now when we have decided to load COBOL sources directly into cobolxom (XML Object Model for a COBOL) the parser did the job perfectly.

The good point about incremental parser is that it easily handles COBOL's grammar.

The whole process looks like this:

1. incremental parser having a COBOL grammar builds a grammar tree;
2. we stream this tree into xml;
3. xslt to transform xml from previous step into cobolxom (TODO).

This is an example of a COBOL:

IDENTIFICATION DIVISION.
PROGRAM-ID. FACTORIAL RECURSIVE.

DATA DIVISION.
WORKING-STORAGE SECTION.
01 NUMB PIC 9(4) VALUE IS 5.
01 FACT PIC 9(8) VALUE IS 0.

LOCAL-STORAGE SECTION.
01 NUM PIC 9(4).

PROCEDURE DIVISION.
  MOVE 'X' TO XXX
  MOVE NUMB TO NUM

  IF NUMB = 0 THEN
    MOVE 1 TO FACT
  ELSE
    SUBTRACT 1 FROM NUMB
    CALL 'FACTORIAL'
    MULTIPLY NUM BY FACT
  END-IF

  DISPLAY NUM '! = ' FACT

  GOBACK.
END PROGRAM FACTORIAL.

And a grammar tree:

<Program>
  <Name data="FACTORIAL"/>
  <Recursive/>
  <DataDivision>
    <WorkingStorageSection>
      <Data>
        <Level data="01"/>
        <Name data="NUMB"/>
        <Picture data="9(4)"/>
        <Value>
          <Numeric data="5"/>
        </Value>
      </Data>
      <Data>
        <Level data="01"/>
        <Name data="FACT"/>
        <Picture data="9(8)"/>
        <Value>
          <Numeric data="0"/>
        </Value>
      </Data>
    </WorkingStorageSection>
    <LocalStorageSection>
      <Data>
        <Level data="01"/>
        <Name data="NUM"/>
        <Picture data="9(4)"/>
      </Data>
    </LocalStorageSection>
  </DataDivision>
  <ProcedureDivision>
    <Sentence>
      <MoveStatement>
        <From>
          <String data="'X'"/>
        </From>
        <To>
          <Identifier>
            <DataName data="XXX"/>
          </Identifier>
        </To>
      </MoveStatement>
      <MoveStatement>
        <From>
          <Identifier>
            <DataName data="NUMB"/>
          </Identifier>
        </From>
        <To>
          <Identifier>
            <DataName data="NUM"/>
          </Identifier>
        </To>
      </MoveStatement>
      <IfStatement>
        <Condition>
          <Relation>
            <Identifier>
              <DataName data="NUMB"/>
            </Identifier>
            <Equal/>
            <Numeric data="0"/>
          </Relation>
        </Condition>
        <Then>
          <MoveStatement>
            <From>
              <Numeric data="1"/>
            </From>
            <To>
              <Identifier>
                <DataName data="FACT"/>
              </Identifier>
            </To>
          </MoveStatement>
        </Then>
        <Else>
          <SubtractStatement>
            <Value>
              <Numeric data="1"/>
            </Value>
            <From>
              <Identifier>
                <DataName data="NUMB"/>
              </Identifier>
            </From>
          </SubtractStatement>
          <CallStatement>
            <Name>
              <String data="'FACTORIAL'"/>
            </Name>
          </CallStatement>
          <MultiplyStatement>
            <Value>
              <Identifier>
                <DataName data="NUM"/>
              </Identifier>
            </Value>
            <By>
              <Identifier>
                <DataName data="FACT"/>
              </Identifier>
            </By>
          </MultiplyStatement>
        </Else>
      </IfStatement>
      <DisplayStatement>
        <Values>
          <Identifier>
            <DataName data="NUM"/>
          </Identifier>
          <String data="'! = '"/>
          <Identifier>
            <DataName data="FACT"/>
          </Identifier>
        </Values>
      </DisplayStatement>
      <GobackStatement/>
    </Sentence>
  </ProcedureDivision>
  <EndName data="FACTORIAL"/>
</Program>

The last step is to transform tree into cobolxom is in the TODO list.

We have commited COBOL grammar in the same place at SourceForge as it was with XQuery grammar. Solution is now under the VS 2010.

Suppose you have a timestamp string, and want to check whether it fits to one of the following formats with leading and trailing spaces:

• YYYY-MM-DD-HH.MM.SS.NNNNNN
• YYYY-MM-DD-HH.MM.SS
• YYYY-MM-DD

We decided to use regex and its capture groups to extract timestamp parts. This left us with only solution: xsl:analyze-string instruction. It took a couple more minutes to reach a final solution:

<xsl:variable name="parts" as="xs:string*">
  <xsl:analyze-string select="$value"
    regex="
      ^\s*(\d\d\d\d)-(\d\d)-(\d\d)
      (-(\d\d)\.(\d\d)\.(\d\d)(\.(\d\d\d\d\d\d))?)?\s*$"
    flags="x">
    <xsl:matching-substring>
      <xsl:sequence select="regex-group(1)"/>
      <xsl:sequence select="regex-group(2)"/>
      <xsl:sequence select="regex-group(3)"/>

      <xsl:sequence select="regex-group(5)"/>
      <xsl:sequence select="regex-group(6)"/>
      <xsl:sequence select="regex-group(7)"/>

      <xsl:sequence select="regex-group(9)"/>
    </xsl:matching-substring>
  </xsl:analyze-string>
</xsl:variable>

<xsl:choose>
  <xsl:when test="exists($parts)">
    ...
  </xsl:when>
  <xsl:otherwise>
    ...
  </xsl:otherwise>
</xsl:choose>

How would you solve the problem? Is it the best solution?