Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Generified Flow Search Algorithms #2

Merged
merged 108 commits into from
Aug 25, 2016
Merged
Show file tree
Hide file tree
Changes from 103 commits
Commits
Show all changes
108 commits
Select commit Hold shift + click to select a range
5f996d5
Initial implementation of predicate-based flow scanner and an increme…
svanoort Apr 14, 2016
ddb608d
Pacify FindBugs
svanoort Apr 14, 2016
2b51497
Tests for the unrefactored FlowScanner
svanoort Apr 18, 2016
bb0864e
WIP on pulling out generalized flowscanning algorithms
svanoort Apr 18, 2016
f4de174
Flesh out/fix most of the flow scanner implementations
svanoort Apr 19, 2016
d414723
Finish tests for FlowScanner algorithms and fix remaining edge cases
svanoort Apr 19, 2016
4c8faa4
Add visitor method to FlowScanner for collecting stats while walking …
svanoort Apr 19, 2016
eaf4769
Fix FindBugs complaints
svanoort Apr 19, 2016
0ed42f8
Flesh out incremental analysis cache skeleton
svanoort Apr 19, 2016
7f4d488
Move FlowScanner method to search by StepDescriptor into test only to…
svanoort Apr 19, 2016
deb585b
Rename findAllMatches to filter
svanoort Apr 25, 2016
72e49df
Refactor flow graph incremental analysis, add stub of a test until I …
svanoort Apr 25, 2016
eb48552
Fix imports in incremental flow analysis
svanoort Apr 25, 2016
e640fd3
Refactor the incremental flow analysis to allow testing by running in…
svanoort Apr 26, 2016
7c1e6df
Remove the incremental flow analysis so we can push it into a separat…
svanoort Apr 26, 2016
6dfbb2d
Start a ForkFlowScanner (WIP), add lots of Javadoc/comments, rename B…
svanoort Apr 27, 2016
ae9581b
Syntactic sugar and comments on FlowScanner
svanoort Apr 27, 2016
1331ddd
FlowScanner: switch to simpler iterator use internally, add filterato…
svanoort Apr 30, 2016
a307266
More AbstractFlowScanner helpers, add iterator on block starts, add t…
svanoort May 2, 2016
211f297
FlowScanning: ForkScanner passes in basic case, next up is testing ne…
svanoort May 2, 2016
c714502
Add standard gitignore items
svanoort May 3, 2016
94857ac
Test ForkScanner blacklisting with parallel branches & fix issues
svanoort May 3, 2016
94845a1
Refactor FlowScanner & affiliated classes to be prepare for review
svanoort May 3, 2016
f4ebc6e
Massive refactor: give graph analysis its own package, and split the …
svanoort May 3, 2016
8e8ddfb
Rename graphanalysis test package
svanoort May 3, 2016
b90e878
Optimize the DepthFirstWalker by non tracking non-BlockStart nodes
svanoort May 3, 2016
b44f402
Save the WIP for ForkScanner with multiple heads
svanoort May 17, 2016
366e080
Add explanation header to AbstractFlowScanner
svanoort May 17, 2016
233107f
Clean up from initial review: add and fix javadocs, refactor some fie…
svanoort May 17, 2016
2bf2829
Small changes from review, and @NonNull annotations on FlowNode
svanoort May 17, 2016
733ab4a
More changes from review
svanoort May 17, 2016
7c31794
Refactor and fixes from last refactor
svanoort May 18, 2016
a1ceb70
Finish stripping down the ForkScanner and the core least-common-ancestor
svanoort May 18, 2016
11be993
Somewhat better behaved version of ForkScanner, just need to fix blac…
svanoort May 18, 2016
b1f0976
Fix blacklisting, remove unused fields of ForkScanner objects
svanoort May 19, 2016
e598a91
Add suggested review API, add a missing NotNull, explain the null han…
svanoort May 24, 2016
7173c9c
LinearFlowScanner test is deterministic
svanoort May 24, 2016
2707059
Add megatest for FlowScanner abstract functionality
svanoort May 25, 2016
21c6623
Harden the flowscanner tests for ordering, clean up
svanoort May 25, 2016
13ec682
Refactor forkscanner, create dedicated tests, fix splitting
svanoort May 25, 2016
4ee5cec
WIP for ForkScanner - tests and 1 fix for leastCommonAncestor
svanoort May 25, 2016
6e28d68
Fix the remaining issues with LeastCommonAncestor core & add comments
svanoort May 31, 2016
0cddee1
Fix unused imports
svanoort May 31, 2016
09a5d21
Fix remaining bugs in ForkScanner, add test, and clean up docs/commme…
svanoort May 31, 2016
5a77748
Tidy up and HTML format the graph analysis javadocs
svanoort May 31, 2016
8bb6e6a
More javadocs + package annotation for graphanalysis
svanoort May 31, 2016
10e2eb4
For graph analysis add a FlowChunk object to use in analysis APIs
svanoort Jun 7, 2016
389543b
For flow scanning, add an in-memory representation of timed blocks
svanoort Jun 8, 2016
bd426b3
Add a parallel flow chunk for duration computation
svanoort Jun 8, 2016
2efcba4
Define interface for chunk storage
svanoort Jun 9, 2016
8ebbdc9
Save simple block visitor
svanoort Jun 10, 2016
f97a08f
Basic forkscanner block API
svanoort Jun 10, 2016
13ef687
Add status handling and memory storage engine to graph analysis
svanoort Jun 10, 2016
094cb36
Carve out advanced visitor methods
svanoort Jun 10, 2016
2116580
Advanced visitor for carving up flows (skeletal impl) and parallel ha…
svanoort Jun 10, 2016
698d0d3
Chunk and block storage types
svanoort Jul 24, 2016
b8ac909
Merge branch 'master' into block-scanning-APIs
svanoort Jul 24, 2016
64ec619
Customize SNAPSHOT version so we can distinguish this from master sna…
svanoort Jul 24, 2016
9efad20
Remove all the parts not needed for raw block scanning
svanoort Aug 5, 2016
1338b64
Formalize the block visitor API with parallels and branches yet again
svanoort Aug 5, 2016
6c4b8e3
Harden ChunkVisitor and start a FlowChunker
svanoort Aug 5, 2016
0237e96
Forkscanner now tracks state that shows what kind of nodes we have (a…
svanoort Aug 8, 2016
abdf296
When splitting chunks you need to know previous node for inclusive vs…
svanoort Aug 10, 2016
a955451
Add some peeking methods to ForkScanner and also a few small access/f…
svanoort Aug 10, 2016
6eb3705
Finish the internal SimpleChunkVisitor iteration logic in ForkScanner…
svanoort Aug 10, 2016
ac2609d
Remove the useless peek methods
svanoort Aug 10, 2016
e8e6203
Fix test failure
svanoort Aug 10, 2016
b5c2b9d
More small fixes
svanoort Aug 10, 2016
866e21d
Test visitor and tests for visitor iteration in ForkScanner
svanoort Aug 11, 2016
cd6e3dd
Soften requirements for MemoryFlowChunk
svanoort Aug 11, 2016
84656e5
Remove unused StandardSimpleChunkVisitor for now
svanoort Aug 11, 2016
55dc020
Add pause timing to MemoryFlowChunk
svanoort Aug 12, 2016
4af21f5
Fixes and test fixes for simple visitor parallels
svanoort Aug 12, 2016
1627155
Add standard visitor and convenience methods for forkscanner/simpleCh…
svanoort Aug 12, 2016
e1988cd
Cleanup for reviews
svanoort Aug 12, 2016
606de64
Merge branch 'master' into generic-flow-analyzer
svanoort Aug 12, 2016
c9f2444
Merge branch 'generic-flow-analyzer' into block-scanning-APIs
svanoort Aug 12, 2016
a53b13e
Remove dangling file
svanoort Aug 12, 2016
aa18ee5
Oh yeah I meant to change the versioning
svanoort Aug 12, 2016
f22aba6
Better reset the pause time for StandardChunkVisitor
svanoort Aug 12, 2016
9880dc8
Rename finder and add the stagefinder that I removed earlier
svanoort Aug 12, 2016
700d009
Remove unused BlockVisitor from early prototypes
svanoort Aug 12, 2016
586979c
Fix minor oopsy
svanoort Aug 12, 2016
7bb581c
Remove StageChunkFinder because BlockScopedStages need some additiona…
svanoort Aug 15, 2016
7b9faca
Cleanup author annotation misformatting for javadoc
svanoort Aug 17, 2016
ae47cc2
Make all paragraph tags in javadocs not self-closing to placate javadocs
svanoort Aug 17, 2016
d21d67a
Placate more javadocs complaints
svanoort Aug 17, 2016
1678b7d
Fix more javadocs complaints and silence doclint nonsense
svanoort Aug 17, 2016
2030360
More exhaustive tests for SimpleVisitor and fix a pair of swapped arg…
svanoort Aug 19, 2016
6b6eba7
Address review comments from @oleg-nenashev
svanoort Aug 19, 2016
33e079e
Fix an off-by-one case in ForkScanner with parallels and add more ext…
svanoort Aug 21, 2016
56e42eb
Restrive access to some of ForkScanner internals
svanoort Aug 21, 2016
94e9169
Remove unused totalBranches field in ForkScanner
svanoort Aug 21, 2016
6cd4215
Address review comments from @oleg-nenashev
svanoort Aug 23, 2016
9d05aa6
Apply a few small review changes
svanoort Aug 23, 2016
d332d8c
Re-enable linting just to see how much pain there is
svanoort Aug 23, 2016
9067f00
Javadocs update
svanoort Aug 24, 2016
8ed03ee
Fix more JavaDocs formatting due to DocLint being a jerk.
svanoort Aug 24, 2016
80b7ad4
Fix yet more doclint whining
svanoort Aug 24, 2016
cf4d5ac
OMG doclint really
svanoort Aug 24, 2016
4f1575c
Address review comments, mostly javadocs
svanoort Aug 24, 2016
0bc9ffe
Merge branch 'block-scanning-APIs' into generic-flow-analyzer
svanoort Aug 24, 2016
4cc9206
Javadoc error.
jglick Aug 22, 2016
2bcfd8b
Solved a javadoc hiccup
svanoort Aug 24, 2016
34483c3
Remove accidentally commit merge files
svanoort Aug 24, 2016
78e8d2c
Revert special versioning in prep for release
svanoort Aug 24, 2016
8abdf34
Fix a missing DepthFirstScanner reset of field
svanoort Aug 24, 2016
3d3bf57
Fix and thoroughly test against a bug with ForkScanner failing when h…
svanoort Aug 25, 2016
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 25 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
@@ -1,4 +1,28 @@
target
work
.idea

# IntelliJ project files
*.iml
*.iws
*.ipr
.idea
out

# eclipse project file
.settings
.classpath
.project
build

# vim
*~
*.swp

# ctags
tags

# OS X
.DS_Store

# mvn versions:set
pom.xml.versionsBackup
22 changes: 20 additions & 2 deletions pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@
</parent>
<groupId>org.jenkins-ci.plugins.workflow</groupId>
<artifactId>workflow-api</artifactId>
<version>2.2-SNAPSHOT</version>
<version>2.2-blockapis-SNAPSHOT</version>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please revert before merging.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jglick Done.

<packaging>hpi</packaging>
<name>Pipeline: API</name>
<url>https://wiki.jenkins-ci.org/display/JENKINS/Pipeline+API+Plugin</url>
Expand All @@ -48,7 +48,7 @@
<developerConnection>scm:git:git@github.com:jenkinsci/${project.artifactId}-plugin.git</developerConnection>
<url>https://github.com/jenkinsci/${project.artifactId}-plugin</url>
<tag>HEAD</tag>
</scm>
</scm>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On my personal TODO list for years:

mojo revision 14519: get maven-release-plugin to use decentxml (also consider org.apache.maven.archetype.common.DefaultPomManager)

In short: this was not a human mistake, and (last I checked) fixes will not “stick”.

<repositories>
<repository>
<id>repo.jenkins-ci.org</id>
Expand All @@ -70,5 +70,23 @@
<artifactId>workflow-step-api</artifactId>
<version>1.15</version>
</dependency>
<dependency>
<groupId>${project.groupId}</groupId>
<artifactId>workflow-job</artifactId>
<version>1.15</version>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🐜 use 2.x for all new dependencies (I am anyway migrating to 2.x for existing dependencies)

<scope>test</scope>
</dependency>
<dependency>
<groupId>${project.groupId}</groupId>
<artifactId>workflow-cps</artifactId>
<version>2.2</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>${project.groupId}</groupId>
<artifactId>workflow-basic-steps</artifactId>
<version>1.15</version>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not 2.x?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No response...

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@oleg-nenashev It did not require a bump -- if we auto-bump to 2.x, it makes sense to do that separately.

<scope>test</scope>
</dependency>
</dependencies>
</project>
Original file line number Diff line number Diff line change
Expand Up @@ -114,13 +114,15 @@ public final boolean isRunning() {
/**
* Returns a read-only view of parents.
*/
@Nonnull
public List<FlowNode> getParents() {
if (parents == null) {
parents = loadParents(parentIds);
}
return parents;
}

@Nonnull
private List<FlowNode> loadParents(List<String> parentIds) {
List<FlowNode> _parents = new ArrayList<FlowNode>(parentIds.size());
for (String parentId : parentIds) {
Expand All @@ -135,6 +137,7 @@ private List<FlowNode> loadParents(List<String> parentIds) {

@Restricted(DoNotUse.class)
@Exported(name="parents")
@Nonnull
public List<String> getParentIds() {
List<String> ids = new ArrayList<String>(2);
for (FlowNode parent : getParents()) {
Expand Down
Loading