This is very timely for something I've been wanting to do! Giving it a try now.
Edit: tried it. I like it, I hadn't used playwright before but it was fairly straightforward to set up the test I wanted. Since you requested feedback, some thoughts:
- I started by writing and testing my check locally, and then copied it over into a “synthetic test”. It would be cool if that were more seamless, e.g. if all of my tests were configured through a git repo and DevRaven had a GitHub webhook that was notified when the config changed.
- I had to double-escape some backslashes in my code for it to be accepted, you might be eating the escapes during some string processing?
- It would be cool to see timings of tests (i.e. the time between individual asserts as reported by playwright, but even the overall time would be a useful metric).
- This is probably just my inexperience with playwright, but it would be nice to have an example that used playwright's expect function instead of chai.
Comments
This is very timely for something I've been wanting to do! Giving it a try now.
Edit: tried it. I like it, I hadn't used playwright before but it was fairly straightforward to set up the test I wanted. Since you requested feedback, some thoughts:
- I started by writing and testing my check locally, and then copied it over into a “synthetic test”. It would be cool if that were more seamless, e.g. if all of my tests were configured through a git repo and DevRaven had a GitHub webhook that was notified when the config changed.
- I had to double-escape some backslashes in my code for it to be accepted, you might be eating the escapes during some string processing?
- It would be cool to see timings of tests (i.e. the time between individual asserts as reported by playwright, but even the overall time would be a useful metric).
- This is probably just my inexperience with playwright, but it would be nice to have an example that used playwright's expect function instead of chai.
- TypeScript support would also be nice.
Thanks Paul for the feedback.
- ya, great idea to be able to pull the changes via git.
- I will look into the issue. edit: confirmed a bug.
- The overall time is captured and is shown in the logs. The dashboard view will be available very soon showing the execution time.
- I will add an example to the recipes for expect as well.
- Agreed on TypeScript support. I will check the feasibility to support on current stack.
update: double-escape issue is now fixed.
That was fast! Cheers.