Should TestCafe's TypeScript support be deprecated?

[DanKaplanSES]

What is your Scenario?

Forgive me because I think this title might appear incendiary, but I write this in good faith with the intent of starting a discussion.

I've been using TestCafe's TypeScript support for a while now. Back when I decided to do so, I didn't know as much about TypeScript as I do now. At the time, it sounded like it would save me a compilation step, and I figured if TestCafe provided first-class typescript support, I should probably use it.

Years later, I have enough experience to realize the trade-offs of that decision, and as far as I can tell, there are more downsides to upsides. Here are some of issues it's caused me and others:

---

TestCafe v2.0 and up includes a TypeScript 4 compiler. TestCafe v1.X includes a TypeScript 3 compiler.

This means the ts compiler you use for your production code might be different from the one in your testcafe tests. This has the potential to cause strange bugs that are difficult to track down; tsc may compile your project fine, but TestCafe reports issues, or vice versa. It limits certain ts features to specific parts of your code base.

---

TestCafe passes the following options to the TypeScript compiler unless you override them:

Option	Value
allowJs	true
emitDecoratorMetadata	true
experimentalDecorators	true
inlineSourceMap	true
noImplicitAny	false
pretty	true
suppressOutputPathCheck	true
skipLibCheck	true

These aren't always aligned with tsconfig defaults. This can make your tsconfig appear strange relative to the typical tsconfig: if you want to remove one of these settings, you add the property.

You can override some but not others. e.g., you can't modify target (at least, I think that's why I errors/warnings about it sometimes). If I'm correct about testcafe being the source of those, I don't think this table is comprehensive: target isn't mentioned.

I don't even know what suppressOutputPathCheck is, as it's not officially documented.

---

VSCode is limited to parsing a root tsconfig.json (at least, that's my interpretation of that linked issue). This is limiting if your production code is typescript in the same code base because you already need a tsconfig.json file for that code, and you may want the same compiler options for both.

---

This is similar to the above, but reusing the same TestCafe tsconfig file as a project reference is challenging, impossible, or at the very least, undocumented by TestCafe. The only workaround I can come up with is a tsconfig file for testcafe's usage and another tsconfig file for vscode's usage. But this is a DRY violation, allowing your tsc/vscode compilation to get out sync with TestCafe's.

---

There are other open issues in their issue tracker.

---

I hesitate to bring this one up because I can't reproduce it, but I experience issues where testcafe uses lowercase paths for declaration files, causing errors because those same paths have already be included with original casing.

---

Anyway, I wrote the above with the intention of being thorough, not to vent. Here's the point I'm trying to get to: as far as I can tell, none of these potential pitfalls occur if you bypass TestCafe's TS support and compile your own TS files instead.

I can only think of one downside to doing that. You have to put additional effort into setting up a TS project to compile your test files. (This is something you need to do anyway if your production code is already written in typescript.) This advantage is documented here:

You do not need to manually compile TypeScript tests. TestCafe does this automatically on launch.

My question: are there other advantages of using TestCafe's TypeScript support I haven't listed?

If not, I think these alternatives have fewer downsides than using testcafe's TS support:

1. Documentation showing how to set up a ts project to run testcafe tests
2. A bare bones, public github project
3. Honestly, if you remove the feature/docs, I don't think there's a need to replace it with anything. If someone wants to write their tests in typescript, I think they'd already expect to learn typescript.

What are you suggesting?

Described above

What alternatives have you considered?

Described above

Additional context

I wanted to ask this as a stackoverflow question titled, "What are the advantages to using TestCafe's typescript support," but I think it would be closed for being subjective.

[aleks-pro]

Hello @DanKaplanSES ,

Thank you for your feedback. We will consider it when we allocate resources to improve TypeScript support. Of course, we will review a Pull Request regarding these issues should there be any. Right now, we are focused on other tasks, so I'll close the issue, but we will still be able to continue the discussion in this thread.

[DanKaplanSES]

Thanks for your reply. I'm still curious about this question: are there other advantages of using TestCafe's TypeScript support I haven't listed?

[aleks-pro]

The advantage of TestCafe's TypeScript support is that you can write tests in typescript without any additional setup. If you want to use a specific TypeScript compiler version or a custom configuration file, you can use customCompilerModulePath and configPath compiler options: https://testcafe.io/documentation/402824/guides/intermediate-guides/typescript-and-coffeescript#customize-compiler-options

[DanKaplanSES]

The advantage of TestCafe's TypeScript support is that you can write tests in typescript without any additional setup.

That can be a timesaver, I agree.

If you want to use a specific TypeScript compiler version or a custom configuration file, you can use customCompilerModulePath and configPath compiler options: https://testcafe.io/documentation/402824/guides/intermediate-guides/typescript-and-coffeescript#customize-compiler-options

Okay but these steps do require additional set up and come with the trade-offs mentioned in the original post. What are the benefits of doing this?

[aleks-pro]

Hello @DanKaplanSES ,

What are the benefits of doing this?

I see this in the following way.
The approach with custom compilation creates a set of javascript files with tests. TestCafe will run this set of files. So, you'll have one set of files in which you write your tests (typescript) and the second set in which you debug your tests (javascript). Possibly, we can overcome this shortcoming via Typescript’s Path Mapping mechanism, but to be sure, I need to research this use case.
On the other hand, TestCafe compiles typescript files with tests on the fly. So, there is no additional set of files with tests. If you take this into account, the fact that you need to set up customCompilerModulePath and configPath compiler options to use a specific TypeScript compiler version or/and a custom configuration file doesn't look like a big deal. But we will consider creating an example illustrating this approach.
Also, there are cases when a project with a test application exists separately from a project with tests.

Okay but these steps do require additional set up and come with the trade-offs mentioned in the original post.

Please create a separate issue for each trade-off you've mentioned. It will help us to process your inquiry more efficiently.

[DanKaplanSES]

Please create a separate issue for each trade-off you've mentioned. It will help us to process your inquiry more efficiently.

Will do.

Does that apply to your latest reply too, or should I continue that conversation here? I'll include it here but will copy it over if that's what you prefer:

---

The approach with custom compilation creates a set of javascript files with tests.

That's a good point I hadn't thought of. Maybe some projects would make it difficult to avoid bundling these js files with the production app.

So, you'll have one set of files in which you write your tests (typescript) and the second set in which you debug your tests (javascript).

When you say "debug," are you referring to testcafe's debug(), VSCode's (or whatever IDE you are using) debugger, debugging the source code within the browser dev tools, the stack traces you see when assertions fail, all of the above, or something else entirely?

Possibly, we can overcome this shortcoming via Typescript’s Path Mapping mechanism, but to be sure, I need to research this use case.

The answer to my previous question might answer this one, but what shortcoming are you referring to; I don't understand what problem might be solved by typescript's Path Mapping.

If you take this into account, the fact that you need to set up customCompilerModulePath and configPath compiler options to use a specific TypeScript compiler version or/and a custom configuration file doesn't look like a big deal.

Let me try to reword this so you can tell me if I'm understanding. Are you saying, "The trade-offs to dealing with the transpiled files are comparable/larger than the trade-offs listed in my original post?"

[Aleksey28]

Hi @DanKaplanSES,

When you say "debug," are you referring to testcafe's debug(), VSCode's (or whatever IDE you are using) debugger, debugging the source code within the browser dev tools, the stack traces you see when assertions fail, all of the above, or something else entirely?

All the cases you listed were mentioned.

The answer to my previous question might answer this one, but what shortcoming are you referring to; I don't understand what problem might be solved by typescript's Path Mapping.

We need additional research to give you a correct answer.

Let me try to reword this so you can tell me if I'm understanding. Are you saying, "The trade-offs to dealing with the transpiled files are comparable/larger than https://github.com/DevExpress/testcafe/issues/7978#issue-1870506976?"

It depends on your trade-offs. For most use cases, it is enough and easy to use our compiler on the fly mechanism because you don't need to perform additional setup steps. As @aleks-pro mentioned, it would be better if you create a separate issue for each trade-off you've mentioned.

[DanKaplanSES]

@Aleksey28 @aleks-pro Done. Let me know if I've misunderstood the request.