Better way of detecting form inputs and submissions

Feature name:

Better way of detecting form inputs and submissions

Feature Description

One of the main issues I’ve had with bitwarden ever since I started using it, is how rare it is for the login auto save feature to actually work. On most sites, including major ones like Google, the notification bar never pops up, so you have to manually remember to add every login. Sometimes the notification bar also disappears when the page refreshes.
I decided to try and fix this, and have come up with new code to detect forms and fields, which in my testing has worked flawlessly when compared to other managers like Lastpass and Dashlane. The only current major problem is forms that aren’t under an actual form element, such as Aliexpress’s login modal. These forms are still detected and the passwords saved, but the input detection fails.
The only thing missing now is to improve the notification bar itself, my plan is to make it a small modal in the corner that allows for reviewing and editing, which comes in handy for forms where the wrong input is detected (i.e. detects email but username is used for login), or miss some data.
I’ve also been thinking of using the new improved input detection (which can distinguish a simple password field from a new password one, for example) to add a small icon next to each input, similar to other manages, that allow for easier access to autofill and password generation. There have been many requests for this, and it would of course be toggleable, just like the current detection and notification bar.
None of the actual autofilling code has been touched, so there should be nothing to worry about that.

Edit: First PR is up at Improve form submission detection by fuwwy · Pull Request #2066 · bitwarden/browser (github.com)

Clients / Repos Affected:

  • Web

Timeline to completion (estimate):

ETA: Q4/2021

7 Likes

Thanks, @Folf!

I think @eliykat and @djsmith85 would be great candidates to take a look at this. If you already have code written, could you make it publically visible on a fork perhaps? Also any screenshots/gifs would be helpful, too :slight_smile:

2 Likes

Hey there, I’m currently tidying up the code and I’ll see if I can get it up on github later today.
Here’s a quick showcase of the auto saving feature working on Facebook, Google and this site, only the first of which works with the current extension, as well as some minor work on the popup icons.
There’s also currently a bug with the folders feature (cc @eliykat) that is preventing the notification bar from working, the folder selection uses a class but the code is looking for an id.

One question that I have currently is when both an username and an email field is displayed, which one should the auto save prioritize?

auto-save
change pw

2 Likes

Hi @Folf

this looks great and I’m looking forward to see what you’ve come up with. The issue with the folder selection not working properly should be fixed and has been merged into master.

Feel free to @ me in the PR once it’s up.

2 Likes

@Folf that notificationBar bug is fixed by the way :slight_smile:

2 Likes

Thank you for this.

It’s been a while since I’ve gone through all the Bitwarden code. If you have a few minutes, would you mind posting a brief synopsis of what you are doing differently than the current Bitwarden implementation?

Also, could the techniques you are using be used to help Bitwarden successfully fill fields on demand? There are a number of sites where Bitwarden won’t fill certain fields. (I’ve filed reports for all the ones I have encountered.)

As for the form submission detection, I wrote a small explanation on the PR which is linked on the main post. Field detection is pretty much done too and it’s mostly about working out the types, attributes and amount of inputs in forms and figuring out what is what (i.e. if there’s more than one password field and if any attribute including current, old, former, or new is found, assume it as a change password form).

I’m yet to touch anything autofilling related, I want to keep most of that code as is to prevent any currently working sites from breaking, but there’s some improvements I want to do, such as not autofilling new password fields. Could you send me a link to these reports?

2 Likes

Some progress
Is there any icon I can use for password generation?

overlays

3 Likes

Hmm - we do have some new icons coming. @Hinton do you have a recommendation?

1 Like

I would suggest using the refresh icon from FontAwesome for now. fa-refresh: Font Awesome Icons

2 Likes

Most of the work is done. I’ve pushed the code on my fork if anyone’s interested.
Right now I’m just giving it some daily usage to see if I can spot any issues.

7 Likes

I’m currently having a major problem with firefox since chrome.extension.getBackgroundPage is not available outside the extension context (so, iframes) which makes the popups fail to grab any of the services. Does anyone have any idea on how to work around this?

Found a link to this thread in Overlay popup interface - #228 by tgreer. Bit confused though. It seems you are working on a better password detection, however I am also seeing a pop-up overlay implementation.

The link in your main posts refers to the branch “feature/better-detection” (8 commits ahead of master). However, I also see that a branch named “feature/popup-overlay” (13 commits ahead of master) exists. It appears “feature/popup-overlay” contains also “feature/better-detection”.

So the main question out of all of this. Which branch to use to try out the pop-up feature?

Hi @aex351 I am not the developer, but I was also interested in testing this new feature so let me share with you my experience regarding build locally:

First of all you need to clone the feature/popup-overlays repo as it contains the code associated with the popup-overlay, and the improved input/submission detection as well. (As you mentioned above.)

According to Folf, there seems to be some issue regarding gulp that’s being fixed on upstream.
So after
0, installing alll the programs needed
1, cloning the repo locally (git clone https://github.com/fuwwy/bitwarden-browser.git)
2, cd into it, and
3, checking out the branch you want to build (git checkout feature/popup-overlays) you need to
4, add upstream as a remote (git remote add upstream https://github.com/bitwarden/browser ),
5, pull the changes (git pull upstream master ), then
6, proceed with npm install and
7, build (I used the command npm run dist, as npm run build:watch didn’t work for me)

After the succesful build you will need to look for the dist-chrome folder and load into chrome (I use it with brave browser, in which it works the same way) as an ‘unpacked extension’

Sadly I don’t know how to proceed if you want to build it for safari/firefox/etc…

Hope I could help :sweat_smile:

3 Likes

We usually use runtime.sendMessage to communicate between content scripts and the extension. runtime.sendMessage() - Mozilla | MDN

Just tried it, works great. I like the pop-up overlay. It is not intrusive and works pretty smooth in Chrome. If you click on the input field you will still get the browser auto-complete function. Clicking on the icon next to it will give you a Bitwarden drop down menu, which looks almost identical to the current browser extension menu from the toolbar.

In regards to your question about Firefox and Safari. If I run npm run dist. I get a dist folder containing various available browser extensions (Chrome, Opera, Firefox and Safari) each in their own zip folder.

1 Like

This is amazing! If this works like the videos show I think this will be enough for me to finally shut down Lastpass. Can’t wait for it to be finished!

4 Likes

Does this still work? I am seeing merge conflicts during build. Thanks

It still works.However, if you are seeing merge conflicts then this means that the upstream project (Bitwarden itself) is becoming out of sync. It could be that the changes from this branch need to get merged. However, it is now end of October and this thread started beginning of September and no one from Bitwarden itself did anything with this.

Be great if someone could merge the 2 code sets and make this work again. :smiley:

1 Like