Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add functionality to build a global flamegraph of implicit searches #50

Merged
merged 6 commits into from
Nov 23, 2023

Conversation

danicheg
Copy link
Collaborator

Resolves #43

This PR brings the -P:scalac-profiling:generate-global-flamegraph plugin option that enables building the global flamegraph of implicit searches for all compilation units. In fact, these graphs are built us assembling the flamegraphs of all compilation units. The Flamegraph tool accumulates all repetitive tokens on its own in the resulting graph.

Comment on lines +3 to +5
object ScalaSettingsOps {
def isScala212: Boolean = true
def isScala213: Boolean = false
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That is a bit controversial, perhaps there is a more elegant way to do these checks.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just for the context, scala/scala#9575.

@SethTisue
Copy link
Collaborator

@lolgab since you made the initial feature request, could we interest you in taking this for a quick spin before merge...?

val names =
stackedNames.getOrElse(id, sys.error(s"Stack name for search id ${id} doesn't exist!"))
val stackName = names.mkString(";")
//val count = implicitSearchesByType.getOrElse(tpe, sys.error(s"No counter for ${tpe}"))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this commented out line needed?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Preserved it as a historical artefact 🤷🏻

def foldImplicitStacks(outputPaths: Seq[AbsolutePath]): Unit =
if (outputPaths.nonEmpty) {
// This part is memory intensive and hence the use of java collections
val stacksJavaList = new java.util.ArrayList[String]()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did you consider using a scala.collection.mutable.ArrayBuffer?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I left this as is (as Jorge implemented it initially). But I also felt that we can change this collection over time.


val globalDir =
ProfileDbPath.toGraphsProfilePath(
config.sourceRoot.resolve(RelativePath(s"target/$scalaDir/classes"))
Copy link
Contributor

@lolgab lolgab Nov 20, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The target directory is customizable in sbt. This doesn't work if you customize it. Maybe the plugin should receive the ScalaDir directory as parameter which opens the door to use the plugin also in other build tools like mill where the directory structure is different.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I thought about this one. If we agree on the approach, I will iterate on this. From my understanding, scalac-profiling was designed for SBT mostly. There are also a few other things related to SBT's output.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In my projects at work I have:

// avoid invalidating compilation cache when enabling/disabling coverage
target := { if (ScoverageKeys.coverageEnabled.value) target.value / "coverage" else target.value }

which will break here. But we don't need to fix this problem now. It can be iterated over later :)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@lolgab but this would also change targets for particular modules, and currently generated flamegraphs will be put into the coverage space, won't it?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. but your hardcoding of target will not work.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So perhaps we need to generalize the task and make target configurable overall, wdyt?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

but your hardcoding of target will not work.

btw, have you tried specifying the source root directory?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can ask for crossTarget and then look inside for the classes.
For example it can be something like: s"-P:scalac-profiling:crossTarget=${crossTarget.value}".
crossTarget points by default to target/scala-2.13 but it can be configured and matches what sbt uses.
If you change target sbt automatically changes crossTarget.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But yeah, it doesn't need to be done in this PR. Just something to keep in mind for later.

@SethTisue
Copy link
Collaborator

SethTisue commented Nov 22, 2023

This is ready for merge, or is it still under discussion...?

@danicheg
Copy link
Collaborator Author

@SethTisue as we agreed with @lolgab, their concern about using scalac-profiling in other than SBT build tools will be resolved within #55. So, this should be good to go.

@SethTisue SethTisue merged commit 529f77d into scalacenter:main Nov 23, 2023
2 checks passed
@danicheg danicheg deleted the fix-#43 branch November 23, 2023 20:16
@danicheg danicheg added the scalac-profiling Relates to the compiler plugin label Dec 10, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
scalac-profiling Relates to the compiler plugin
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Generate flamegraph for the entire project merging multiple modules
3 participants