Issue using svn with svn+ssh:// protocol

[shawn99452]

Our SVN instance is configured using svn+ssh:// protocol; everyone connects using a public key. We use virtual users - everyone specifies the same linux user on the svn server, and the authorized_keys file on the server automatically maps the private key used to log in to the appropriate user using the --tunnel-user option. Normally, from the command line, we would use something like "svn checkout svn+ssh://SVN_USER@DOMAIN_NAME/project/trunk". On the web server, I've created a virtual user for Apache's user with read-only access to svn; using "sudo -u www-data svn ls SVN_PATH/@, I get a working listing of the remote repository, so mantis should work with it. I've tried configuring the URL similarly in SourceSVN, and get the error listed below. I've also tried omitting the user, and specifying it in the "svn user" section (leaving the password blank since we use keys), and that also fails (with a different less informative error).

I'm guessing that this isn't working because this is a fairly niche use of svn (we do this so we can have the repository externally available for remote work, without having to worry about credentials being guessed; normal user/password authentication is disabled), and to simplify deployment scripts on remote servers that pull from svn.

I've modified mantis a few times to fix issues and extend it, and am proficient in linux administration tasks, and am a senior full-stack software developer, so I'm ok with making code changes to make this work, but I wanted to see if anyone else had tried with a configuration like this, and I just don't have it set up correctly, or had pointers on where to start looking in the plugin to make this work right.

Error message when I try to use Import Latest Data, or Import Everything; happens immediately (I've replaced our svn user with SVN_USER below, and our domain name with DOMAIN_NAME):

APPLICATION ERROR #plugin_SourceSVN_svn_cmd

Subversion execution returned an error: "svn: E205000: Try 'svn help info' for more information
svn: E205000: Syntax error parsing peg revision 'DOMAIN_NAME'; did you mean 'svn+ssh://SVN_USER@DOMAIN_NAME@'?".
Please use the "Back" button in your web browser to return to the previous page. There you can correct whatever problems >were identified in this error or select another action. You can also click an option from the menu bar to go directly to a new >section.

[shawn99452]

Haven't verified this at all, but from the command line, if you do something like "svn ls svn+ssh://SVN_USER@DOMAIN_NAME/", you get an error; you have to specify an extra @ at the end of the command, if the last portion of the path is a directory and not a file. If you don't do this, svn interprets the first @ separating the username and domain name as a revision peg. I think this happens with any path where you use the "user@host/path" format with the svn binary, it's not specific to svn+ssh protocol.

[shawn99452]

Looks like that is the problem. Not a good solution, but appending "@" to $t_url in SourceSVN.php's import_full() allows the command to work properly. Fixing it properly would probably require always appending the @ to every URL that's not supplying a revision peg, so all the calls to svn_run() would have to be modified manually, or it would have to be modified to construct the path differently.

There is a secondary problem, it only parses up to revision 199 and doesn't actually import any changesets (our current revision number is 269xx) so it must be hitting some error parsing the results. Will have to look into that separately and make another issue if I figure out if it's an issue with the plugin or something wrong with our repository. But this issue still needs fixing - constructing the repo path using a function seems like a not-too-hard way to fix the issue, since there aren't THAT many svn commands it runs.

[dregad]

@shawn99452 Thanks for the report and analysis, and sorry about delayed response.

Unfortunately I can't really be of much help to you here as I do not use SVN, but if you are willing to submit a pull request, I'll gladly review and merge it.

[dregad]

There is a secondary problem, it only parses up to revision 199

There is a hardcoded --limit 200 parameter in the call to svn log [1] which could be the reason for this ?

[shawn99452]

Yes, that looks like it would cause that on a new install, since $t_rev will be unset. That's pretty crazy though, as it will never work right for anyone with any reasonable activity in their repository unless they've always used the plugin, so $t_rev is reasonably up-to-date. My guess is that the limit is due to the way the data gets processed - it sticks it in a big string variable, which obviously won't work if you have 27000 revisions because it will take forever and/or exceed PHP's memory limits. It really should be using popen() or something here to read the data into one of PHP's xml libraries that supports streams, like XMLreader, or at the very least use a loop and pull revision ranges in chunks until it has read and imported everything. It probably would still have to use some kind of background worker for this, or use ini_set() to change the max_execution_time so it doesn't get killed after 60 seconds. So that the user doesn't just sit there confused why the page isn't loading, a background worker that gets spawned on import would make the most sense, so the plugin could display an 'import progress' page, with a progress bar that does ajax requests to get the progress from the worker.

This is of course a huge refactor of the plugin, and while I could do this, I'm not sure if I will - I was just looking into whether there was an easy way to integrate SVN and Mantis; I'm not unwilling to make minor changes to support our use cases, but this is a pretty big change for not a whole lot of improvement, so it can't be justified as a thing I do for my work. Maybe I'll do it on my own time at some point...

[dregad]

I don't use SVN anymore myself, so there's just no way I'd start working on such refactoring either. Maybe someone from the community will be willing to take it on.