Bug #6780

Model TransferableGraph exports can write wrong statement count into the exported file

Added by Tuukka Lehtonen over 1 year ago. Updated over 1 year ago.

Status:ClosedStart date:2016-09-02
Priority:4Due date:2016-09-02
Assignee:Tuukka Lehtonen% Done:

100%

Category:ExportSpent time:-
Target version:1.22.1.3 service release
Release notes:Fixed major bug in TransferableGraphs.writeTransferableGraph that caused the code to write an invalid statement count into the resulting TG file in cases where the original data contained statements that would be thrown away as garbage during the final transferable graph file writing process. The written statement count failed to take into account the discarded statements which could make it larger than it is in reality. When this bug was triggered, it effectively made the exported file unimportable.
Tags: export, major, transferable-graph
Story points-
Velocity based estimate-
Release Simantics 1.22.1.3 service releaseRelease relationshipAuto

Description

A follow-up regression caused by the changes done in #6668 has been discovered. There is a subtle but nasty bug in TransferableGraphs.writeTransferableGraph that causes the code to write an invalid statement count into the resulting TG file.

The problem is that TransferableGraphSource.forStatements is currently coded so that it may still filter out some of the statements that are written into the temporary list of exported statements. This was not taken into account in the mentioned writeTransferableGraph function, which just wrote the original statement count into the file. This essentially makes the file invalid and unreadable. Manifestation of this problem requires that the exported model/data has so called garbage statements that the forStatements invocation will filter out. If the exported data does not contain such garbage, then everything is fine.

Note that the statement count that in problem cases is written is simply too large and that this is the only problem with the file - one single integer that has a wrong value. We can write a heuristic program that can automatically fix these problems by finding the correct value for the statementCount and overwriting it from the file.

Another thing to note is that the potential for this bug to manifest has been in the platform since the changes done in #3454. Those changes contained logic that would have been capable of changing the amount of exported statements during the execution of TransferableGraphSource.forStatements but that logic seems to not have been triggered that much since then. The changes and mistakes done in #6668 really only caused these old problems to surface.


Related issues

Related to Platform - Bug #3454: TG file streaming Closed
Follows Platform - Bug #6668: ModelTransferableGraphSource writes invalid statements in... Closed 2016-09-01 2016-09-01

Associated revisions

Revision 33348
Added by Tuukka Lehtonen over 1 year ago

Fixed major bug in TransferableGraphs.writeTransferableGraph that caused the code to write an invalid statement count into the resulting TG file.

The problem is that TransferableGraphSource.forStatements is currently coded so that it may still filter out some of the statements that are written into the temporary list of exported statements. This was not taken into account in the mentioned writeTransferableGraph function, which just wrote the original statement count into the file. This essentially makes the file invalid and unreadable. Manifestation of this problem requires that the exported model/data has so called garbage statements that the forStatements invocation will filter out. If the exported data does not contain such garbage, then everything is fine.

Note that the statement count that in problem cases is written is simply too large and that this is the only problem with the file - one single integer that has a wrong value. We can write a heuristic program that can automatically fix these problems by finding the correct value for the statementCount and overwriting it from the file.

refs #6780

Revision 33349
Added by Tuukka Lehtonen over 1 year ago

Merged /ode/branches/simantics-1.22.1 r33348 /ode/{trunk,branches/simantics-1.22}.
Merged /foundation/databoard/branches/simantics-1.22.1 r33348 /foundation/databoard/{trunk,branches/simantics-1.22}.

refs #6780

Revision 3efd7d44
Added by Tuukka Lehtonen over 1 year ago

Sync git svn branch with SVN repository r33349.

refs #6600
refs #6774
refs #6780

Revision 0cbe7783
Added by Tuukka Lehtonen over 1 year ago

Merge commit '3efd7d44a55ae1fff284a1826e7c9afe27e0815e'

refs #6600
refs #6774
refs #6780

History

#1 Updated by Tuukka Lehtonen over 1 year ago

  • Status changed from In Progress to Feedback

#2 Updated by Tuukka Lehtonen over 1 year ago

  • Target version changed from 1.22.1.2 service release to 1.22.1.3 service release

#3 Updated by Tuukka Lehtonen over 1 year ago

  • % Done changed from 0 to 100
  • Release changed from 44 to 46
  • Release notes set to Fixed major bug in TransferableGraphs.writeTransferableGraph that caused the code to write an invalid statement count into the resulting TG file in cases where the original data contained statements that would be thrown away as garbage during the final transferable graph file writing process. The written statement count failed to take into account the discarded statements which could make it larger than it is in reality. When this bug was triggered, it effectively made the exported file unimportable.

#4 Updated by Tuukka Lehtonen over 1 year ago

  • Description updated (diff)

#5 Updated by Tuukka Lehtonen over 1 year ago

  • Description updated (diff)
  • Status changed from Feedback to Closed

Also available in: Atom PDF