Vote #65498
未完了Data Import/Export System
0%
説明
Hi All,
I think it will be very useful to have some kind of data import/export system for Redmine with the following features:
Entire projects could exported or imported to different instances of Redmine. I know this is dependent on user information being included and restored with the project in some capacity.
Selectively export/import administrative settings including user information. This one probably needs to be smart enough to allow a controlled data merge to take place during an import.
This could serve also as the basis of a backup system for various areas of Redmine
I got the idea as I was deleting a test project. If you can delete a project in its entirety, you should be able to archive all of that information before deletion in one simple step as the delete vs copy logic is pretty much the same. And if you can do that and then delete your project, you should be able to take that exact same archive and restore it.
I haven't investigated this at the code level but wanted to post this because it seems a good idea at this moment and a search of data export did not provide any interesting hits
journals
+1
Good idea.
Also it can be a simple way to make backup.
--------------------------------------------------------------------------------
I agree this would be great, and just what we need for a number of cases.
This relates to http://www.redmine.org/boards/2/topics/5754#message-5761, where just this functionality is required.
--------------------------------------------------------------------------------
I would love to help out if I can. I am not a coder. My skill set is more geared to project management and testing/debugging. Let me know if I can assist.
--------------------------------------------------------------------------------
+1
Recently I received a request to Import only a specific project from our test server to PROD. (MySQL Server)
I did it manually exported records from table by table & imported them in PROD. Took few hours for about 100 Issues.
It will be nice to have this feature to export a single or multiple project data from one server & import to another RedMine server.
--------------------------------------------------------------------------------
+1
Yes, it would be very nice to export/import projects and administration/user settings. Good for backup, moving between servers :)
--------------------------------------------------------------------------------
+1
--------------------------------------------------------------------------------
+1
We have 3 redmine server... And now i try to merge it into one server...
Server 1: 14 project [100+ issues]
Server 2: 3 project [100+ issues]
Server 3: 8 project [100+ issues]
Pls make a migration tool...
/* mysql_dump is not a good way: 'issueId = (1..100+) is in all server */
thx
--------------------------------------------------------------------------------
+1
--------------------------------------------------------------------------------
+1
--------------------------------------------------------------------------------
+1
--------------------------------------------------------------------------------
+1
--------------------------------------------------------------------------------
+1
--------------------------------------------------------------------------------
+1
--------------------------------------------------------------------------------
+1
--------------------------------------------------------------------------------
+1
--------------------------------------------------------------------------------
+1
--------------------------------------------------------------------------------
+1
--------------------------------------------------------------------------------
Raito Yitsushi wrote:
> +1
>
> We have 3 redmine server... And now i try to merge it into one server...
> Server 1: 14 project [100+ issues]
> Server 2: 3 project [100+ issues]
> Server 3: 8 project [100+ issues]
>
> Pls make a migration tool...
>
> /* mysql_dump is not a good way: 'issueId = (1..100+) is in all server */
>
> thx
dfdf
--------------------------------------------------------------------------------
+1
--------------------------------------------------------------------------------
+1
--------------------------------------------------------------------------------
+1
The real problem relates to the issues IDs (and in particular the fact that these IDs can be used in SVN commit messages to reference the issue). If Redmine had project-specific issue IDs (for example: shortName-1234), it would be simple to import a project.
With global server IDs, I fear that the correct import of a project will not be possible (because that will change the IDs, so at least the references on SVN commits will be lost or wrong).
However, it's a real problem !
--------------------------------------------------------------------------------
Thomas Zlika wrote:
> +1
> The real problem relates to the issues IDs (and in particular the fact that these IDs can be used in SVN commit messages to reference the issue). If Redmine had project-specific issue IDs (for example: shortName-1234), it would be simple to import a project.
> With global server IDs, I fear that the correct import of a project will not be possible (because that will change the IDs, so at least the references on SVN commits will be lost or wrong).
> However, it's a real problem !
mmm....
1. If you have two different SVN repositories you must first of all merge them, maintaining all the history and creating a report with OldCommitReference->NewCommitReference. This is not a easy task, in any case it is related only to SVN. With such a report you have to update the RedMine servers renaming oldReference with newReference wherever it was used (wiki, notes, etc).
2. Once you merge two RedMine Servers you need a report with OldIssueID->NewIssueId. With that report you have to update all the Commit Comments in your SVN repository, replacing old issue id with new issue id as described in the "Subversion FAQ":http://subversion.apache.org/faq.html#change-log-msg (the Import process could do that automatically). You need to replace the issue IDs not only in SVN but also in all the notes, wiki of RedMine.
Sure, with a global issue ID (like with GIT commits) it would be easier to merge two servers, but the simple incremental Issue ID number is more human friendly....
--------------------------------------------------------------------------------
+1 This would be useful, as I often projects that have started off in my own Redmine later require more access and need a Redmine of their own.
--------------------------------------------------------------------------------
It's nearly impossible to have a clean answer to this question. Something could be built on top of @yaml_db@ gem (https://github.com/ludicast/yaml_db ; it generates a database-independent export in YAML). But the latest posts are right on issue IDs, and I can't imagine such a change in a near future sorry.
--------------------------------------------------------------------------------
Dear Jean-Baptiste,
thank you for taking the time to comment.
Okay, it is technically hard (nearly impossible) to provide this feature, so we'll have to leave without it (unless somebody volunteers to pay [a lot] for the development work).
Suppose several projects grow on one Redmine instance, and one of them needs to be transferred to a new server. Could one clone the whole system, and then on the cloned system delete all projects apart from the desired one? This is not pretty, but at least a workable solution available now.
Would we expect this to work?
Thank you,
Hans
--------------------------------------------------------------------------------
Hans: indeed ! It can be a solution if your projects/users are strongly separated from each other. It could even be partially scripted.
Anyway, if you experience load problems :
# you might be able to solve it outside of Redmine, with a caching reverse-proxy, multiple passenger/mongrel instances (or even a different hosting solution), etc.
# you can report any specific problem here, I think we would be happy to help Redmine be better with high traffic loads
--------------------------------------------------------------------------------
A couple things:
I also saw some code for a tentative way to do this over here: http://www.redmine.org/boards/1/topics/8077
Also, the unique id thing is the main problem definitely. I think there are a couple ways to possibly handle this:
1) start down the path of UUID's for objects. This would also have the benefit of allowing people to choose between a system wide unique issue number or a project wide unique issue number. This could possibly be done with the help of a plugin, but would more likely be best implemented at the core of redmine. See http://en.wikipedia.org/wiki/Universally_unique_identifier for some background. Drupal is a project I know that has some implementations of this http://drupal.org/project/uuid but almost all programming languages have at least one library for using them (ruby has 3-4 i think)
2) use a plugin to manage the mapping of old id's to new id's and then supersede the use of the old id's when linking between version control, issues, wikis, etc.
For example you import issue #253 from site1.com and it becomes #12 on site2.com... when rendering the a link to issue #253 on site2.com, based on per-project settings, it knows to link to issue #12 instead. (The plugin would probably have to create a custom field on the relevant object to store the old id's)
This method could work relatively easily but would probably be a bit hacky. There might need to be a way to resolve/choose which reference you're talking about at times.
--------------------------------------------------------------------------------
--------------------------------------------------------------------------------
+1
--------------------------------------------------------------------------------
+1
--------------------------------------------------------------------------------
Could you please provide me updates about this feature?
I want to transfer redmine data from one machine to another machine.Is there any solution to perform this?
--------------------------------------------------------------------------------
+1
Just built a new server for production and would like to import a project from my old test server. Over 100 issues
--------------------------------------------------------------------------------
+1
(it seems there is no easy way to switch from sqlite3 to mysql, so a data import/export feature could take care of that.)
--------------------------------------------------------------------------------
+1
I need merge several redmine servers (with different databases) in one
--------------------------------------------------------------------------------
Marcelo Soares Souza wrote:
> I need merge several redmine servers (with different databases) in one
I haven't thought of that, but now that you mention it:
Being able to import/export single projects would mean way better data portability, thus also more peace of mind hosting-wise. You could start of a little redmine on your dev box with sqlite3 knowing that if demand gets out of hand you can switch to a 'managed' hosted (like plan.io).
It is already the case with wordpress(.org/.com) and their implementation works really well.
--------------------------------------------------------------------------------
+1
--------------------------------------------------------------------------------
+1
--------------------------------------------------------------------------------
Can't it be done in this way:
All issue IDs will be extended internally by the according project ID. The projectID can be written in links, but is not needed in all cases.
The syntax for issue links (e.g. in wiki) will be extended by someting like #projectID:issueID
and alternatively with previous syntax #issueID referencing to local project (if #issueID occurs) else to other project (if #issueID occurs there). This means that the issueID will be specific only for 1 project in future (not for complete redmine).
So when a project is imported to a different redmine installation all issues will keep their numbers and will stay linked to the actually imported project. Just links to different projects need to be broken, if the according project is not imported in the same time.
Attachments need to be stored in a own folder per project.
Users will be created as new users, or (if name is same in old and new redmine) will be mapped accordingly.
All needed trackers will be transfered as well as workflows, news, wiki and forums.
Also if not all information is transferred it will be better than nothing.
This feature is really needed, if you use redmine for many projects (and therefore on some installations).
--------------------------------------------------------------------------------
I don't like the idea of non-unique issue ID's.
We have two instances of Redmine currently, and are looking to merge these. Already staff use the issue ID to point people to the relevant issue, and with two instances this can be very confusing. If the issue ID was only relevant when mentioned in the context of the project there would be chaos! Different business units use different projects, with their work going into subprojects of that B.U. Project space. There is enough cross-functional/cross project team work going on that noone would know what issue 123 meant anymore (we are lazy enough to leave off the project identifier).
--------------------------------------------------------------------------------
Holger Kluge wrote:
>Hi,
>i also thought about this and stumbled about http://www.rubyrep.org. They have a >screencast for bidirectionally synchronizing two redmine instances at >http://www.rubyrep.org/screencast.html.
>I did not test this myself, but it may be worth a try...
Why don't they sync the two tables 'schema_migrations' and plugin_shema_info'?
Does that mean i can have different plugins setups in both redmine instances?
If redmine standard tables are never touched by plugin migration, can this work?
--------------------------------------------------------------------------------
+1
--------------------------------------------------------------------------------
+1
--------------------------------------------------------------------------------
+1 It looks like these guys figured out a way. http://www.hostedredmine.com/news/500
--------------------------------------------------------------------------------
John NWW wrote:
> +1 It looks like these guys figured out a way. http://www.hostedredmine.com/news/500
I manage HostedRedmine.com. I would be willing to release this code under GNU Affero General Public License. It is PHP code, but it is simple enough to use and modify.
> This would be useful, as I often projects that have started off in my own Redmine later require more access and need a Redmine of their own.
This is the main use case. It allows you to extract out particular users from a Redmine installation and all their associated data (issues, wikis, etc). It must be imported into a fresh installation of Redmine, due to unique issue numbering etc.
--------------------------------------------------------------------------------
Terence Mill wrote:
> Holger Kluge wrote:
>
> >Hi,
>
> >i also thought about this and stumbled about http://www.rubyrep.org. They have a >screencast for bidirectionally synchronizing two redmine instances at >http://www.rubyrep.org/screencast.html.
>
> >I did not test this myself, but it may be worth a try...
>
> Why don't they sync the two tables 'schema_migrations' and plugin_shema_info'?
>
> Does that mean i can have different plugins setups in both redmine instances?
> If redmine standard tables are never touched by plugin migration, can this work?
have no idea about the tables but the synchronization as i see could have been done with a normal mysql cluster. the fun thing would be to do asynchronous bi-directional sync but for that one need to change stepping of how sequence indexes are generated in redmine.
--------------------------------------------------------------------------------
+1
--------------------------------------------------------------------------------
Brad Rushworth wrote:
> John NWW wrote:
> > +1 It looks like these guys figured out a way. http://www.hostedredmine.com/news/500
>
> I manage HostedRedmine.com. I would be willing to release this code under GNU Affero General Public License. It is PHP code, but it is simple enough to use and modify.
>
> > This would be useful, as I often projects that have started off in my own Redmine later require more access and need a Redmine of their own.
>
> This is the main use case. It allows you to extract out particular users from a Redmine installation and all their associated data (issues, wikis, etc). It must be imported into a fresh installation of Redmine, due to unique issue numbering etc.
Hey Brad did you get a chance to release it under GNU? I am in a similar need. Please point me to the location if you published it.
--------------------------------------------------------------------------------
Sudhanshu Agrawal wrote:
> Hey Brad did you get a chance to release it under GNU? I am in a similar need. Please point me to the location if you published it.
Yes, it is now attached here: http://www.hostedredmine.com/news/500
--------------------------------------------------------------------------------
+1
Hello,
there you can find an importer for "RequisitePro" baselines: https://github.com/gen2thomas/redmine_import_reqpro
"RequistePro" is using UID's. This ID's will be imported as custom field "RPUID".
Baselines are human readable xml files with special names and format (schemes). Relations between issues inside a project and also to other projects are supported currently.
The "DevelopmentV05"-Branch is working quite good. I have now imported more than 20000 issues with that plugin into my redmine database.
If anyone can write the same for export in such a baseline format 3 guys will be happy ;o}:
1.) The one who want to use his redmine datas in "RequistePro"
2.) That guy like you, who want to merge some redmine databases together
3.) Me, because I want to clean up my database for all the stuff I have added while testing the importer
Wiki and other associated data are not present in "RequisitePro". For that to export an separate file is possible which will be ignored for importing in "RequisitePro".
--------------------------------------------------------------------------------
+1
--------------------------------------------------------------------------------
+1
--------------------------------------------------------------------------------
+1
In the meanwhile, those who need a guide, I've compiled a bunch of instructions about data migration and upgrade:
"Installing, Migrating & Upgrading Redmine with LDAP on Ubuntu":http://vault-tec.info/post/68670739052/installing-migrating-upgrading-redmine-with-ldap-on
I hope this may be of help to some of you. For me it took about a week to figure all this out lol.
--------------------------------------------------------------------------------
+1
--------------------------------------------------------------------------------
I've found an importer tool (https://github.com/leovitch/redmine_importer/wiki). Maybe it could be used with Brad's exporter (http://www.redmine.org/issues/3647#note-44) somehow?
--------------------------------------------------------------------------------
@Kamil, looks like that importer tool is about 2 years old.. If you don't have any luck getting it to work with the Redmine 2.x range, you might want to try "this one":https://github.com/juanmf/RedmineIssueImporter.
--------------------------------------------------------------------------------
--------------------------------------------------------------------------------
+1
--------------------------------------------------------------------------------
--------------------------------------------------------------------------------
Fredrik Svensson wrote:
> have no idea about the tables but the synchronization as i see could have been done with a normal mysql cluster. the fun thing would be to do asynchronous bi-directional sync but for that one need to change stepping of how sequence indexes are generated in redmine.
The database system will take care of that. E.g. MySQL will issue even ID's on member1 and odd on member2. This works for >2 members, too.
See MySQL Multimaster replication and auto_increment_increment.
--------------------------------------------------------------------------------
This is also related to #950 which although it only covers imports, has just been committed and will be released in 3.2.0.
--------------------------------------------------------------------------------
What ever happened with this?
It'd be really nice to have a way to export the data in a 'clean' form that didn't just dump the entire database.
For example, I have an old, heavily upgraded Redmine instance and I want to start fresh, but import all the data without taking across the schema and any weird tables that various plugins over the years may have left lying around.
--------------------------------------------------------------------------------
> It'd be really nice to have a way to export the data in a 'clean' form that didn't just dump the entire database.
FWIW: A few months ago, I had a similar problem: One project on an instance with about 100 projects needed to be extracted and moved to a new server. At that time, I only needed to deal with this project's Wiki (no issues and forum), so I decided to copy the elements via MySQL. The Perl scripts are on GitHub [1].
This may not be what you are looking for or you may need to synchronize issues too. Unless something better comes along - that's why I track this issue here - I will use it again for the next migration task.
* [1] https://github.com/gonter/redmine-sync
Cheers!
GG
--------------------------------------------------------------------------------
+1, IMHO very important feature... I've tested plugins but it's not the same, now I'll test @Gerhard Gonter's link... Thanks!
--------------------------------------------------------------------------------
I created this which might help some people: https://github.com/delatbabel/redmine-migrator
--------------------------------------------------------------------------------
--------------------------------------------------------------------------------
+1
--------------------------------------------------------------------------------
+1
--------------------------------------------------------------------------------
+1k
--------------------------------------------------------------------------------
+1
--------------------------------------------------------------------------------
I've created https://gitlab.com/meschenbacher/redmine-migration and successfully migrated two projects with a few hundret and a few thousand issues on redmine version 4.1.2.
--------------------------------------------------------------------------------
related_issues
relates,Closed,17259,Import_Button
relates,Closed,7884,Import issues from Excel or CSV file
relates,New,25625,Turn issue numbers into UUIDs
duplicates,Closed,9993,Export and import all information (Database)