[Pattern Draft] Internal Developer Platform

[caldeirav]

Proposing the Internal Developer Platform under the maturity 1-Initial to start getting contributions across the InnerSource Commons.

The pattern will be presented and discussed in our upcoming community call: "Promoting InnerSource Practices with an Internal Developer Platform"

[welcome]

💖 Thanks for opening this pull request! 💖 The InnerSource Commons community really appreciates your time and effort to contribute to the project. Please make sure you have read our Contributing Guidelines.

If you are submitting a new pattern, the following things will help get your pull request across the finish line! 🏁

• Confirm that you have used our pattern template. Please remove any placeholder text and sections that your pattern did not need.
• We run a number of automated checks on your PR. Please review the output of those checks on the PR itself, and see if any issues got flagged that you can fix yourself.
• Make sure you have added your new pattern to the list of patterns in the main README.md. If you are unsure where to add your pattern, just let us know by commenting on your PR and we will help you.

This project has a small number of maintainers, volunteering their time to this project. So please be patient and we will get back to you as soon as we can. If we don't acknowledge this pull request after 7 days, feel free to chat to us about it in our Slack workspace.

[spier]

hi @caldeirav . Thank you for sharing this pattern with us, even including a sketch and everything! Amazing start!

The pattern contains a lot of detail about how the IDP should be structured, and implemented.

What I am missing is a stronger connection to InnerSource. As in, what are the type of InnerSource challenges that the IDP helps to solve?

For concepts like an IDP that are already pretty widely adopted (e.g. Backstage and all), we sometimes face this challenge when trying to describe the impact of the concept as a pattern. We want to clarify whether the concept is "a useful Engineering practice in general" or if it is specifically supporting some of the areas that InnerSource tries to solve for.

Some of the typical reasons for applying InnerSource can e.g. be found here.

Please don't get me wrong, I am not saying that the IDP concept can not be an InnerSource pattern. It is likely more about identifying (and highlighting in the pattern) how this connects to InnerSource.

If it helps I could also try to highlight individual sections in the pattern where I feel we could try to connect it more clearly to InnerSource? Please let me know if that would help.

[caldeirav]

here

Hi @spier, thank you, it is always helpful to have third party reviews because from my perspective I suspect a lot of the "connections" you describe between the IDP practice and InnerSource happened organically through experience. I would definitely like to make this clearer for readers.

First and foremost, we just had a community talk on the subject (recording now available here) and we shared about the symbiotic relationship between IDP and InnerSource from two angles:

1. An Internal Developer Platform is an essential component to scale InnerSource strategies and practices across the organisation at some point. In large organizations, not standardizing development practices makes it very hard for InnerSource contributors to be effective at contributing to new projects across a large number of teams.
2. Conversely, InnerSource practices can supercharge your IDP journey through channeling internal contributions to platform engineering requirements which are common blueprints to address problems faced across the organization.

But let's tie this to some of the concrete challenges referred to in the link you shared here:

Reduce development silos caused by excessive ownership culture
An IDP breaks down silos by standardizing tools, environments, and processes across teams. It fosters InnerSource practices, where code and resources are shared across teams, reducing excessive ownership over particular tools or systems. By providing a centralized platform, IDPs make it easier for teams to collaborate, access shared resources, and contribute to each other’s work without the traditional territorial boundaries​.

Increase innovation speed by reducing time spent solving similar issues by fostering healthy code reuse
IDPs provide a central repository of templates, services, and reusable components, which encourages code reuse across the organization. Developers can access these assets directly from the platform, reducing duplication of effort. By integrating with documentation, APIs, and existing solutions, IDPs streamline the process of leveraging previous work, enabling teams to focus on innovation rather than reinventing the wheel​.

Increase development speed by better cross-team collaboration
IDPs offer shared tools and environments, facilitating cross-team collaboration by providing transparency into what other teams are working on and how they’re solving problems. Teams can easily contribute to or adopt each other’s work, aided by a self-service portal that enables them to set up environments and access services independently. This reduces communication overhead and accelerates collaboration​.

Solve project/team dependencies beyond "wait it out" and "build workarounds", thereby reducing engineering bottlenecks
IDPs allow teams to self-serve infrastructure, environments, and tools, reducing dependency on other teams to manually provision resources. By centralizing access to infrastructure and automating common processes like CI/CD pipelines, dependencies are less likely to create bottlenecks.

Increase quality
IDPs increase software quality by enforcing standardized practices and providing built-in guardrails for governance, security, and compliance. Automated testing and deployment pipelines ensure that all code adheres to the organization’s quality standards before it goes into production. Moreover, since developers can focus more on solving high-level problems rather than fighting with tooling and environment setup, they have more time to improve code quality​.

Increase employee happiness
As highlighted in our presentation, increasing Developer happiness is the number one objective of an IDP (we talked about Platform-as-a-Product). By reducing friction in the development process, an IDP allows developers to focus on what they do best which is writing code and solving meaningful problems. Self-service capabilities reduce waiting times for resources, while centralized platforms eliminate the complexity of managing disparate tools. This leads to a more streamlined developer experience, which in turn boosts job satisfaction and productivity​.

Increase success of new hires
IDPs simplify the onboarding process by providing new hires with easy access to pre-configured environments, documentation, and templates. Rather than spending their first weeks figuring out how to set up their development environment or learning which tools to use, new hires can start contributing much faster. The IDP serves as a single point of entry for everything they need, dramatically reducing ramp-up time​. As discussed during the presentation, developer onboarding time (we measure it typically by looking at the first approved commit) is key metric in IDP implementation.

Build actionable documentation
An IDP integrates documentation directly into the development workflows via self-service portals. Teams can access API documentation, code examples, and templates from the same place they access tools and environments. This centralization ensures that documentation is kept up-to-date and easily accessible. Furthermore, by encouraging InnerSource contributions into the documentation itself (from users), teams can continuously improve documentation based on real-world usage​.

I would definitely like to have your advice on 1) Whether the above makes sense and clarifies your ask (or if you still see gaps) and 2) What would be the best way to integrate the additional information into the pattern template?

And also accessorily what the best way is to get this pull request through while still getting feedback from potential contributors. Thank you for your feedback!

[rmarting]

Hi!

I would like to share some thoughts about this new pattern.

I agree with @spier that reading the pattern is not easy to connect with InnerSource, as most of the benefits are related to software delivery lifecycles, standardization procedures or self-services for developers. Those things can be done applying InnerSource or not.

On the other hand, thinking about InnerSource patterns, I found IDPs as a good way to implement other very well-know pattern: InnerSource Portal. The InnerSource portal can provide that centralized space to share across the organization and teams the different services, products, or repositories to attract contributors or build a community around them. So, an IDP can be the way to implement that pattern, including the benefits described by @caldeirav in the previous comment.

So, I see the InnerSource Portal as the pattern and IDPs as the implementation of that pattern.

Could IDPs be included in the [solutions and implementations] section as another implementation? It should require to summarize this content to fit into that pattern.

Does it make sense? What do you think?

[NewMexicoKid]

typo: Stremline --> Streamline

[spier]

Suggested change

	- Stremline development workflow and improved developer efficiency through self-service workflows and standardized environments.
	- Streamline development workflow and improved developer efficiency through self-service workflows and standardized environments.

[caldeirav]

Thanks for highlighting. Corrected with the last commit.

[caldeirav]

Hi!

I would like to share some thoughts about this new pattern.

I agree with @spier that reading the pattern is not easy to connect with InnerSource, as most of the benefits are related to software delivery lifecycles, standardization procedures or self-services for developers. Those things can be done applying InnerSource or not.

On the other hand, thinking about InnerSource patterns, I found IDPs as a good way to implement other very well-know pattern: InnerSource Portal. The InnerSource portal can provide that centralized space to share across the organization and teams the different services, products, or repositories to attract contributors or build a community around them. So, an IDP can be the way to implement that pattern, including the benefits described by @caldeirav in the previous comment.

So, I see the InnerSource Portal as the pattern and IDPs as the implementation of that pattern.

Could IDPs be included in the [solutions and implementations] section as another implementation? It should require to summarize this content to fit into that pattern.

Does it make sense? What do you think?

@rmarting thanks for reviewing! Here are my replies:

• On connecting better the pattern description with InnerSource, I have already agreed this is required in this very thread. I have described above how they relate to each other in particular the dual relationship between IDP and Inner Source, and have also described 8 challenges in Inner Source adoption that the IDP pattern is helping to solve. If it makes sense to all I can integrate this content in a new draft.
• On your other question, I don't think IDP could or should be included in the InnerSource portal. The portal is generally a component of the IDP and frankly not the most important one (although probably the most visible) and I routinely come across organisations with Developer Portal that have absolutely no Inner Source practice or plan to implement Inner Source in their organisations (in fact probably still the majority).

The "secret sauce" of the IDP is actually the orchestration layer that helps to define / encapsulate organisational practices and make them more consistent across the organisation. Scaling an Inner Source program beyond a few projects / streams is extremely difficult without this type of platform engineering capability. My understanding of patterns is that they can and probably should include best practices of implementing and scaling Inner Source i.e. the relationship between Inner Source and the pattern can very well be asymmetric (in this case, Inner Source requires IDP for scale, but IDP does not necessarily imply Inner Source).

I hope this makes sense and maybe if it is still not obvious just let me know if rewriting with the above in mind can help make the relationship more obvious. My organisation is actually working on several implementations (early stage) as we speak so I am pretty convinced we will have some actual examples / references to validate this pattern in the future.

[caldeirav]

@spier @rmarting I have tried to include our discussions / address the comments into the revised pattern write up. Please have a look and let me know your thoughts.

[rmarting]

@caldeirav Good job! The pattern is now more clearly connected to InnerSource values and principles. I liked the way is described and connected. For my side, looks great!

I would like to spread this pattern to more people to confirm it as a valid pattern. During last InnerSource Summit 2024 there were different sessions related to IDPs, Platform Engineering, Developer Satisfaction and InnerSource as the way to spread knowledge and increase collaboration between teams. IMHO those sessions are evidences confirming IDP as an InnerSource pattern.

Maybe @mcobby, @dterol23 , @wasadigi and @ebdabbs want to share their ideas about this new pattern. And they could think adding them as known instances or acknowledgments of this pattern.

What do you think?

[rmarting]

It could be good add a References chapter to link this pattern with others that could be related, such as:

• InnerSource Portal

Maybe others could be listed here too. Ideas?

[spier]

Sure, why not. I think we already have some other patterns that have used a References section like this.

You can also include links to other patterns as part of the regular text. i.e. if there is an area in this pattern where we naturally talk about the relationship between IDP and InnerSource Portal, we could add the link there.

[spier]

@rmarting I am trying to get this pattern merged, so that it can more easily be linked to. Therefore we might not add this link to the InnerSource Portal now.

However we can leave this comment thread open if you like, so that we might remember it when we improve the pattern in the future (e.g. when we add Known Instances etc).

[spier]

I haven't had time yet to review the latest version of this. Really sorry about that :(

However I noticed this talk at the recent ISC Summit:
https://www.youtube.com/watch?v=INp1BhIkiMA&ab_channel=InnerSourceCommons

I especially found the quote "InnerSource as an IDP catalyst" relevant to this pattern (around minute 7:08 in the video).

Maybe phrasing it the other way around, "IDP as an InnerSource Catalyst", would make for a great title for this pattern?

[spier]

Thank you for your great work on this so far @caldeirav.

I think the content is great already, and we should merge this swiftly now to make it available in our repository, so that other organizations can more easily find this content.

I left some specific suggestions inline, which I hope should be straightforward to work in.

Once that is done please let me know, so that I can get this merged.

After that let's discuss how to find orgs (Known Instances) that also see this "symbiotic relationship between the IDP and InnerSource" and are already using this pattern.

PS: I was trying to remove the two .DS_Store files but don't have permissions to push to your branch. Could you remove those?

[spier]

Suggested change

	- Stremline development workflow and improved developer efficiency through self-service workflows and standardized environments.
	- Streamline development workflow and improved developer efficiency through self-service workflows and standardized environments.

[spier]

Sure, why not. I think we already have some other patterns that have used a References section like this.

You can also include links to other patterns as part of the regular text. i.e. if there is an area in this pattern where we naturally talk about the relationship between IDP and InnerSource Portal, we could add the link there.

[spier]

I broke these up into bullets, so that readers can use this like a checklist.

I did some rewording, however I tried to keep the intended meaning. Please check if it worked :)

Suggested change

	This pattern emerges in organizations with multiple InnerSource development teams working on different projects. As teams grow and evolve, discrepancies in the setup of development environments and access to resources become more prominent, hampering collaboration, introducing security risks, and reducing developer productivity. The need for uniformity without compromising flexibility drives the adoption of an IDP.
	- The organization has development teams working on different projects. These teams are already using InnerSource practices.
	- As teams grow and evolve, discrepancies in the setup of development environments and access to resources become more prominent, hampering collaboration, introducing security risks, and reducing developer productivity.
	- The teams (as well as the leadership of the organization) are looking to increase consistency in tools and processes without compromising on the flexibility to adapt to project-specific needs.

Question:

What do you mean by "access to resources"? An example could help here.

[spier]

In your ISC community talk you mention that 99% of the work has to happen before setting up the IDP itself. i.e. self-service functionality for the developers has to be developed first, before it can be exposed through an IDP.

Therefore I think we should capture these pre-conditions here, in the form of additional elements of the Context section.

For example:

• the organization has a DevEx team (aka platform development team), tasked to improve developer productivity by reducing friction from suboptimal tools and processes
• at least some self-service style functionality has already been developed by the DevEx team

[spier]

The Palet should stay pretty short (max 2 sentences describing problem and solution). Like the shortest possible summary of what this pattern is about.

Therefore we should move this extra text somewhere else in the pattern.

For the time being you could move this to the "Rationale" section, what do you think?

[spier]

While this seems a bit too long for a Patlet, I suggest to leave it as is for now. We can adapt this once we add more Known Instances and feedback from other orgs, before making this pattern available in our online book. (maturity level "Structured")

[spier]

Hi @caldeirav, and happy new year :)

I just wanted to follow-up on this pattern again.

I cleaned up the PR as much as I could on my own. The remaining things I cannot do, as I cannot push to main in your fork.

If you have time, it would be great if you could review the last 3-4 inline comments. Hopefully they won't be that much work for you to resolve.

If you don't have time to work on this, just let me know. In that case I can finish this on my own (by creating a new branch/PR that I can push to).

Thanks again for sharing your idea with us here!