From 425382a318054f359355c060d70bb4b6c45b8a16 Mon Sep 17 00:00:00 2001 From: Google Code Exporter Date: Thu, 30 Apr 2015 11:58:02 -0400 Subject: [PATCH] Migrating wiki contents from Google Code --- GitHubExporterFAQ.md | 105 ++++++++++++++++++++++++++++++++++++++ IssueExporterTool.md | 101 ++++++++++++++++++++++++++++++++++++ IssueMirror.md | 23 +++++++++ MigratingToBitbucket.md | 30 +++++++++++ MigratingToGitHub.md | 49 ++++++++++++++++++ MigratingToGitLab.md | 37 ++++++++++++++ MigratingToSourceForge.md | 27 ++++++++++ ProjectHome.md | 26 ++++++++++ WikiToMarkdownTool.md | 90 ++++++++++++++++++++++++++++++++ 9 files changed, 488 insertions(+) create mode 100644 GitHubExporterFAQ.md create mode 100644 IssueExporterTool.md create mode 100644 IssueMirror.md create mode 100644 MigratingToBitbucket.md create mode 100644 MigratingToGitHub.md create mode 100644 MigratingToGitLab.md create mode 100644 MigratingToSourceForge.md create mode 100644 ProjectHome.md create mode 100644 WikiToMarkdownTool.md diff --git a/GitHubExporterFAQ.md b/GitHubExporterFAQ.md new file mode 100644 index 0000000..b8c446f --- /dev/null +++ b/GitHubExporterFAQ.md @@ -0,0 +1,105 @@ +If you have a question that isn't answered here, please log an issue in the [issue tracker](https://code.google.com/p/support-tools/issues/list) or [contact Google](mailto:google-code-shutdown@google.com) for assistance. + + + +# Questions # + +## Where did my Google Code wikis go? ## +The wikis of projects exported to GitHub are converted into Markdown and placed in new `wiki` branch in the GitHub project's repo. + +If you would rather use the GitHub wikis feature instead, GitHub user [morgant](https://github.com/morgant/) wrote [a tool](https://github.com/morgant/finishGoogleCodeGitHubWikiMigration) to move the wiki files from the `wiki` branc into the repo's wiki section. +## How can I export private issues? ## +Issues on Google Code can be labeled in such a way that they are private (e.g. those with `Restrict-View-*`). Private issues will be migrated to GitHub as if they were deleted. None of their contents will be migrated. + +If you need to export private issues, you should not use the Google Code Exporter, and instead [manually export](https://code.google.com/p/support-tools/wiki/MigratingToGitHub) your project to GitHub. + +As part of that process, you will use Google Takeout to get a full archive of your projects issues, which includes those marked as restricted. And the [Issue Exporter](https://code.google.com/p/support-tools/wiki/IssueExporterTool) tool will allow you to import them to GitHub. + +## Are all projects available to be exported to GitHub? ## +No. Some projects are blocked from automatic export to GitHub, specifically those that have too many issues (1,000+) or those which have already moved. + +In these situations, you will need to [manually export](https://code.google.com/p/support-tools/wiki/MigratingToGitHub) your project to GitHub. + +## Can I migrate a Google Code repository to a GitHub Organization? ## +No, however you can quickly transfer the repository after the export is complete. + +For more information about transferring a repository on GitHub, see [this document](https://help.github.com/articles/transferring-a-repository/). + +# Post-Export Actions # + +After the project gets migrated to GitHub, sometimes additional clean-up work is necessary. + +## Fixing author information (SVN only) ## + +While Git and Mercurial based projects store the author information as an email address, SVN does not. This can lead to strange situations when the project is on GitHub. Commits by `larry.page@gmail.com` would show up in GitHub as being by `larry.page` after the export. (Since that is how SVN stored the information.) This means that the commits won't get linked to the GitHub user with email address `larry.page@gmail.com`. + +You can fix this however by updating the git repo's author information. + +The process for rewriting git history to change an author's email address is documented on GitHub under [changing author information](https://help.github.com/articles/changing-author-info/). + +## Setting the "Project Moved" Flag ## + +Once a project has been successfully exported to GitHub, you will want to update your project's homepage on Google Code to avoid confusion. + +Some project owners simply update their project homepage's text to indicate it has moved to GitHub. For example [subtext](https://code.google.com/p/subtext/) or [bwapi](https://code.google.com/p/bwapi/). + +Another option is to set the Google Code "project moved" flag. When set, attempts to access the project will take users to an interstitial page indicating the new project location. In the future, the page will automatically redirect to well-known project hosting services such as [github.com](https://github.com). + +To set the "project moved" flag, navigate to your project's advanced admin page, at `https://code.google.com/p//adminAdvanced`. + +Once there, enter the new project home page URL under "project moved". For example: https://github.com/google/kythe. Finally click the "Project Moved" button. + +If you need to "un-move" a project, you can go back to the `/adminAdvanced` page at a later time and click "Publish". This will restore your project to its original state, before the move. + +# Known Issues # + +This section covers a few common errors you might run into. If you encounter any other bugs or unexpected conversion issues, please log issues in the [issue tracker](https://code.google.com/p/support-tools/issues/list). + +## Error "GitHub code import failure: ..." ## + +There are several kinds of GitHub import failures which can arise, such as + + * Error! There was an error pushing commits to GitHub. + * Error! There was an error importing commits. + +This can happen for several reasons, the most common one is that the Google Code repository is too large. + +GitHub will block pushes where a single file is larger than 100MiB. Similarly, the default limit for a GitHub is 1GiB. See [working with large files](https://help.github.com/articles/working-with-large-files/). + +Simply `git rm`-ing a 100+MiB file from your repo is not enough, since the large file will still exist in your repos history. Instead, you need to erase the file from your repo's entirely history. (Literally rewrite history.) GitHub provides documentation for how to do this at [removing sensitive data](https://help.github.com/articles/remove-sensitive-data/). + + +If you do have a consistent error importing your code into GitHub please contact GitHub's support at https://github.com/contact. Please include the name of the target repository (e.g. https://github.com/user/repo), and they will be able to assist you from there. + +## Error "GitHub code import took too long." ## + +If the GitHub import process takes too long the project export is terminated. Please retry your project export, and if the problem persists contact [Google](mailto:google-code-shutdown@google.com) or [GitHub](https://github.com/contact?form%5Bsubject%5D=Google+Code+Export:+Error+code+import+took+too+long). + +## Error "Project cannot be migrated because it has too many issues." ## +There is a hard limit on the number of issues that can be exported by the tool, per repo. If your project has more than 1,000 issues the export will fail. + +To migrate your issues repos, you will need to use the stand-alone [Issue Exporter](https://code.google.com/p/support-tools/wiki/IssueExporterTool). + +## Error "Error migrating issues." ## + +There was an error migrating your projects issues. You can try re-exporting your project in GitHub (deleting the repo, and then using the Exporter tool again). + +If the problem persists, please report it in the [issue tracker](https://code.google.com/p/support-tools/issues/list). Be sure to include the name of the Google Code project you are trying to export. + +## Error "Internal Server Error." ## + +Aw snap! Sometimes gremlins work their way into the system. We do our best to stamp them out, but with Google's free food it is hard to avoid snacking after midnight... + +If you see this error someone will be looking into it. Wait a while and retry your project export again. + +## Error "GitHub code import failure." ## + +This error occurs when the GitHub code import API fails. Usually this issue is transient. But if it persists, contact Google or GitHub. + +## Error "Project already exists." ## + +Projects will be migrated to GitHub creating a repo with the same name as the Google Code project. It is not possible to specify a different name for the GitHub repo. + +If your GitHub account already has a repo with the name of the project you wish to export, you will be prohibited from migrating that project from Google Code. + +To work around this, you can temporarily rename your GitHub repo by going to the repo's settings. \ No newline at end of file diff --git a/IssueExporterTool.md b/IssueExporterTool.md new file mode 100644 index 0000000..531492e --- /dev/null +++ b/IssueExporterTool.md @@ -0,0 +1,101 @@ +If you are migrating a project to Bitbucket or GitHub, the Issue Exporter tool can help you migrate your Google Code issue tracker items to your project’s new home. + +If you are interested in migrating your issues to [gitlab.com](http://gitlab.com), a [separate tool](https://gitlab.com/o9000/google-code-to-gitlab) has been developed. (Though not tested or supported by Google.) + +# Install Python # +First you need to [install Python](https://www.python.org/downloads/), though it should be already installed on most Mac and Linux machines. + +Next, [clone the support-tools Google Code project](https://code.google.com/p/support-tools/source/checkout) (i.e. _this_ project's source code). + +``` +git clone https://code.google.com/p/support-tools/ +``` + +# Use Google Takeout to Get Issue Data # +With the Python runtime installed, next we need to download a dump of all of your project's issues. To create an archive of all of your Google Code projects' issues, head to Google Takeout: +https://www.google.com/settings/takeout + +Google Takeout allows you to export project data from many Google Services, but for now we are only interested in "Google Code Project Hosting". + +Google Takeout will create an archive of all of your project’s issues. Note that this is only available for projects which you are an owner of. You will not be able to use Google Takeout to export archives of issues for other open-source projects. + +Depending on the number of Google Code projects you own, and the number of issues they have, exporting an archive may take a very long time. Google Takeout will notify you when your archive is ready. + +Once you have your issue archive, extract the file named `GoogleCodeProjectHosting.json`. This is a JSON dump of all of your projects' issues. (The issues for multiple Google Code projects will be stored in that single file.) + +We will then run a few scripts on this data dump to upload your issues to another location. There are instructions are different for exporting issues to GitHub or Bitbucket. + +# Exporting to GitHub # +Next we will run a script, `github_issue_convert.py`, to upload your project’s issues to a GitHub project. + +## Personal Access Token ## +The next step is to create a personal access token. This is a special password that you will use for this process. When the migration is complete, you can revoke this password. + +Create a personal access token by visting https://github.com/settings/applications and create a new "Personal Access Token". Be sure to check the "public\_repo" scope. (Which authorizes this access token to modify your public repos.) + +## Running the Script ## +You are now all set to migrate your issues to GitHub. + +The script has several parameters: +``` + usage: github_issue_converter.py + [-h] + --github_oauth_token GITHUB_OAUTH_TOKEN + --github_owner_username GITHUB_OWNER_USERNAME + --github_repo_name GITHUB_REPO_NAME + --issue_file_path ISSUE_FILE_PATH + --project_name PROJECT_NAME +``` + +The parameters are as follows: + + * `github_owner_username` is the name of the GitHub repo owner, i.e. you. (e.g. “chrsmith”) + * `github_oauth_token` this is the access token you generated above. + * `github_repo_name` is the name of the repository you will be importing the issues into. + * `issue_file_path` is the file path to your GoogleCodeProjectHosting.json file. + * `project_name` is the name of the Google Code project whose issues you are exporting. For example "support-tools". + +To run the conversion script, run the following: +``` +python \ + googlecode-issues-exporter/github_issue_converter.py \ + --github_oauth_token="$GH_ACCESS_TOKEN" \ + --github_owner_username= \ + --github_repo_name= \ + --issue_file_path= \ + --project_name= +``` + +### Speeding Things Up ### +Exporting issues and comments to GitHub will be _very, very_ slow. This is because the script is designed to send requests slow enough to not run afoul of GitHub's anti-abuse rate limiting. + +You can get the anti-abuse rate limited for your account by [contacting GitHub](https://github.com/contact?form%5Bsubject%5D=Google+Code+Export:+API+Abuse+Rate+Limits). If you get the rate limit raised for your account, add a `--rate_limit=false` flag to the GitHub issue converter. + +For more information please see the GitHub APIs [section on rate-limiting](https://developer.github.com/v3/#rate-limiting), and search for "Abuse Rate Limits". + +# Exporting to Bitbucket # +Exporting issue data to Bitbucket is slightly different. Rather than using an API to export issues individually, Bitbucket takes a single archive and will process all issues at once. + +We will run a script, `bitbucket_issue_convert.py`, to convert your Google Code issues dump into a format that can be imported into Bitbucket. + +The Bitbucket script is very similar to the GitHub import script above. + +``` +python googlecode-issues-exporter/bitbucket_issue_converter.py \ + --issue_file_path= \ + --project_name= \ + --default_owner_username= +``` + +This will generate the file named `db-1.0.json`, which contains your project’s issues and comments in a format that Bitbucket can ingest. + +Next you need to convert it to a zip file: + +``` +zip -r bitbucket-issues.zip db-1.0.json +``` + +## Upload the Bitbucket Issue Dump ## +Open up your project on Bitbucket, and on the "Settings" page head to "Import & Export" under "Issues". + +Select `bitbucket-issues.zip` file and click "Start import". Done! Now you should have all of your Google Code issues in Bitbucket. \ No newline at end of file diff --git a/IssueMirror.md b/IssueMirror.md new file mode 100644 index 0000000..e9f8dbb --- /dev/null +++ b/IssueMirror.md @@ -0,0 +1,23 @@ +# Issue Mirror # + +As part of the export process, either to GitHub or using the [Issue Exporter](IssueExporterTool.md) tool, issue attachments are presented as a link. This wiki describes how those links are generated. + +All _public_ Google Code issue attachments are mirrored to [Google Cloud Storage](https://cloud.google.com/storage/), in the bucket `google-code-attachments`. + +You can access a project's issue attachments by generating a URL based on the issue's information. The format is as follows: + +``` + "http://storage.googleapis.com/google-code-attachments" + + "/" + project_name + + "/issue-" + issue_number + + "/comment-" + comment_number + + "/" + file_name +``` + +The end result will be a URL to the project's issue, for example: + +http://storage.googleapis.com/google-code-attachments/issue-export-test/issue-4/comment-1/misc_file2.txt + +If an attachment is added to the initial issue report, then it is considered to be at `comment_number` 0. + +Note: There was a bug in an earlier version of the Issue Mirror, which caused comment numbers for issues mirrored to Google Cloud Storage to match Google Code. This only occurred if some of the comments on the issue were deleted, which occurred for < 5% of issue attachments. This issue has since been fixed. \ No newline at end of file diff --git a/MigratingToBitbucket.md b/MigratingToBitbucket.md new file mode 100644 index 0000000..43202f1 --- /dev/null +++ b/MigratingToBitbucket.md @@ -0,0 +1,30 @@ +[Bitbucket](https://bitbucket.org) has been providing both public private source hosting to the internet since 2008. This document will cover how to migrate a project on Google Code to Bitbucket. However, Bitbucket only supports the Mercurial and Git protocols. SVN-based projects can be imported, but will be converted as part of the migration. + +# Sign in to Bitbucket # +If you do not already have a Bitbucket account, create one and log in. You do not need to create a new Bitbucket project before importing your Google Code project; when you start the importer tool, the Bitbucket project will be created for you. + +# Create a new Project via Import # +Once you have logged into Bitbucket, simply click Create to begin creating a new repository and then click the "importing a repository" link. + +That will lead you to a simple wizard for importing projects. Select "Google Code" from the Source combo box, and then enter your Google Code project name. You can find more information in the [Bitbucket Import documentation](https://confluence.atlassian.com/display/BITBUCKET/Import+code+from+an+existing+project). + +The Bitbucket importer may take a few minutes to import your project, depending on the size of your repo. + +Note that the "Issue tracking" and "Wiki" checkboxes are for whether or not you want your Bitbucket project to have the Issue tracking and Wiki _features_. Bitbucket will not import your Google Code issues or wikis. You will need to run separate tools to export that data from Google Code. + +# Project Issues # +To export project issues to Bitbucket, see the documentation for the [Issue Exporter](IssueExporterTool.md). + +# Project Wikis # +To export project wikis to Bitbucket, see the documentation for the [Wiki-to-Markdown Tool](WikiToMarkdownTool.md). + +# URL Redirection # +Once a project has been successfully exported to Bitbucket, you will want to update your project's homepage on Google Code to avoid confusion. + +Some project owners simply update their project homepage's text to indicate it has moved. For example [subtext](http://code.google.com/p/subtext) or [bwapi](http://code.google.com/p/bwapi). + +Another option is to set the Google Code "project moved" flag. When set, attempts to access the project will take users to an interstitial page indicating the new project location. In the future, the page will automatically redirect your new project on bitbucket.org. + +To set the "project moved" flag, navigate to your project's advanced admin page, at https://code.google.com/p/project-name/adminAdvanced. + +Once there, enter the new project home page URL under "project moved". For example: https://bitbucket.org/larryp/search_engine. Finally click the "Project Moved" button. \ No newline at end of file diff --git a/MigratingToGitHub.md b/MigratingToGitHub.md new file mode 100644 index 0000000..d9e7051 --- /dev/null +++ b/MigratingToGitHub.md @@ -0,0 +1,49 @@ +[GitHub](https://github.com) has been providing both public and private source hosting since 2008. This document will cover how to migrate a Google Code project to GitHub. + +Note that while Google Code will host source code using the SVN, Mercurial, or Git protocol, GitHub chiefly supports Git. Mercurial or SVN-based Google Code projects will be converted to the Git protocol as part of the migration. + +# Project Source # +There are several options for importing your source code into GitHub, depending on the type of source control system you use. The rest of this post will cover how to migrate git-based projects to GitHub. But if you are using another VCS, such as SVN or Mercurial, see this [GitHub article](https://help.github.com/articles/importing-your-project-to-github/). + +## Using Git ## +To migrate your project's source to GitHub using command-line tools, first use your GitHub account to create a new, empty repository. Next, add the new GitHub repo as a "remote" to your Google Code project. You can see the git remote originally pointing to Google Code by typing `git remote -v`. + +To move the source code to your new GitHub project, simply change the "origin" remote to point to GitHub. + +``` +$ git remote set-url origin \ + https://github.com/chrsmith/.git +$ git remote -v +origin https://github.com/chrsmith/.git (fetch) +origin https://github.com/chrsmith/.git (push) +``` + +Next, push your changes to the new remote: + +``` +git push -u origin --all # push the repo and its refs +git push -u origin --tags # push any tags +``` + +Once the data has been pushed, your project's source code and history should all be available on GitHub. + +If your git push fails for some reason, for example "error: git-remote-https died of signal 13" run `git fsck` to diagnose the problem. GitHub also provides documentation for [Importing a Git repository using the command line](https://help.github.com/articles/importing-a-git-repository-using-the-command-line/). + +With the project source migrated, the next step is to bring over Google Code project's issues and wikis. + +# Project Issues # +To export project issues to GitHub, see the documentation for the [Issue Exporter](IssueExporterTool.md). + +# Project Wikis # +To export project wikis to GitHub, see the documentation for the [Wiki-to-Markdown Tool](WikiToMarkdownTool.md). + +# URL Redirection # +Once a project has been successfully exported to GitHub, you will want to update your project's homepage on Google Code to avoid confusion. + +Some project owners simply update their project homepage's text to indicate it has moved. For example [subtext](http://code.google.com/p/subtext) or [bwapi](http://code.google.com/p/bwapi). + +Another option is to set the Google Code "project moved" flag. When set, attempts to access the project will take users to an interstitial page indicating the new project location. Future requests to the Google Code projects will automatically be redirected to the new location on github.com. + +To set the "project moved" flag, navigate to your project's advanced admin page, at https://code.google.com/p/project-name/adminAdvanced. + +Once there, enter the new project home page URL under "project moved". For example: https://github.org/larryp/search_engine. Finally click the "Project Moved" button. \ No newline at end of file diff --git a/MigratingToGitLab.md b/MigratingToGitLab.md new file mode 100644 index 0000000..d412e91 --- /dev/null +++ b/MigratingToGitLab.md @@ -0,0 +1,37 @@ +[GitLab](http://gitlab.com/) is a service that hosts git source code repositories, along with wikis and issue tracking. But unlike other services such as GitHub, you can also run GitLab on your own infrastructure. This document will cover how to migrate a project from Google Code to GitLab. + +# Sign in to GitLab # +You can sign-into GitLab using an !OpenID identity, such as your Google account. From you will see the import from Google Code option available on the "New Project" page. + +But first, we need to get the issue tracker data for your Google Code project. + +# Obtain your Google Code Issue data # + +Project migration to GitLab is keyed off of the Google Code Project Hosting archives returned from Google Takeout. To create an archive of all of your Google Code projects' issues, head to Google Takeout: https://www.google.com/settings/takeout + +Google Takeout allows you to export project data from many Google Services, but for now we are only interested in "Google Code Project Hosting". + +Google Takeout will create an archive of all of your project’s issues. Note that this is only available for projects which you are an owner of. You will not be able to use Google Takeout to export archives of issues for other open-source projects. (And consequently you cannot migrate those to GitLab.) + +Depending on the number of Google Code projects you own, and the number of issues they have, exporting an archive may take a very long time. Google Takeout will notify you when your archive is ready. + +Once you have your issue archive, extract the file named `GoogleCodeProjectHosting.json`. This is a JSON dump of all of your projects' issues. (The issues for multiple Google Code projects will be stored in that single file.) + +Upload the .json file to GitLab as part of the project migration. + +# Migrate your Project(s) # + +GitLab's New Project flow guides you through each step of the migration process. With the list of projects obtained from your JSON dump, simply click "Import" for any projects you want to have migrated to GitLab. + +As an optional step, GitLab provides is the ability to map Google Code email addresss to another name. For example, convert the obfuscated "johnsmi...@gmail.com" from the `GoogleCodeProjectHosting.json` dump to a GitLab user, or a full name such as "John Smith". + +# URL Redirection # +Once a project has been successfully exported to GitLab, you will want to update your project's homepage on Google Code to avoid confusion. + +Some project owners simply update their project homepage's text to indicate it has moved. For example [subtext](http://code.google.com/p/subtext) or [bwapi](http://code.google.com/p/bwapi). + +Another option is to set the Google Code "project moved" flag. When set, attempts to access the project will take users to an interstitial page indicating the new project location. In the future, the page will automatically redirect your new project on gitlab.com. + +To set the "project moved" flag, navigate to your project's advanced admin page, at https://code.google.com/p/project-name/adminAdvanced. + +Once there, enter the new project home page URL under "project moved". Finally click the "Project Moved" button. \ No newline at end of file diff --git a/MigratingToSourceForge.md b/MigratingToSourceForge.md new file mode 100644 index 0000000..a86e4be --- /dev/null +++ b/MigratingToSourceForge.md @@ -0,0 +1,27 @@ +[SourceForge](http://sourceforge.net/) has been hosting open-source software since 1999, and boasts over [400,000 projects](http://sourceforge.net/about). This document will cover how to migrate a project from Google Code to SourceForge, which supports the SVN, Git, and Mercurial version control systems. + +The migration will be done using SourceForge's [Google Code Project Importer](https://sourceforge.net/p/import_project/google-code/). + +# Sign in to SourceForge # +If you do not already have a SourceForge account, create one and log in. + +# Navigate to the Importer Tool # +Navigate to the easy-to-use Google Code Project Importer: +https://sourceforge.net/p/import_project/google-code/ + +This tool will import your project’s wikis, issues, existing downloads, and, of course, your project’s source code. You can learn more specifics by reading the tool’s [documentation](http://sourceforge.net/p/forge/documentation/Google%20Code%20Importer/). + +You do not need to create a new SourceForge project before importing your Google Code project; when you start the importer tool, the SourceForge project will be created for you. + +Once you click Import, the process will start. You will get emails as various import steps complete, such as when all the wiki pages have been migrated. + +# URL Redirection # +Once a project has been successfully exported to SourceForge, you will want to update your project's homepage on Google Code to avoid confusion. + +Some project owners simply update their project homepage's text to indicate it has moved. For example [subtext](http://code.google.com/p/subtext) or [bwapi](http://code.google.com/p/bwapi). + +Another option is to set the Google Code "project moved" flag. When set, attempts to access the project will take users to an interstitial page indicating the new project location. In the future, the page will automatically redirect your new project on sourceforce.net. + +To set the "project moved" flag, navigate to your project's advanced admin page, at https://code.google.com/p/project-name/adminAdvanced. + +Once there, enter the new project home page URL under "project moved". For example: https://sourceforge.net/p/fuego. Finally click the "Project Moved" button. \ No newline at end of file diff --git a/ProjectHome.md b/ProjectHome.md new file mode 100644 index 0000000..80b34bf --- /dev/null +++ b/ProjectHome.md @@ -0,0 +1,26 @@ +This project contains tools for exporting data from Google Code project hosting. + +# Google Code to GitHub Exporter # +The quickest way to export data from Google Code is to use the [Google Code-to-GitHub exporter](http://code.google.com/export-to-github/). This tool allows you to export any public Google Code repository to your GitHub account. + +For more information, please see the [Exporter FAQ](GitHubExporterFAQ.md). + +# Project Migration # +There are guides for migrating Google Code projects to other open-source hosting services: + + * [How to migrate a project to GitHub](MigratingToGitHub.md) + * [How to migrate a project to Bitbucket](MigratingToBitbucket.md) + * [How to migrate a project to SourceForge](MigratingToSourceForge.md) + * NEW [How to migrate a project to GitLab](MigratingToGitLab.md) + +# Issue Tracker # +The Google Code Issue Tracker has been integrated with Google Takeout. There is a stand-alone tool for exporting an archive of a Google Code project's issues and importing them into GitHub or Bitbucket. + +See [Issue Exporter](IssueExporterTool.md). + +For information specifically about issue attachments, see [Issue Mirror](IssueMirror.md). + +# Wiki Data # +Google Code's wiki format is non-standard. There is a stand-alone tool for converting Google Code's wiki format to GitHub-flavored Markdown. + +See [Wiki-to-Markdown tool](WikiToMarkdownTool.md). \ No newline at end of file diff --git a/WikiToMarkdownTool.md b/WikiToMarkdownTool.md new file mode 100644 index 0000000..8c191d0 --- /dev/null +++ b/WikiToMarkdownTool.md @@ -0,0 +1,90 @@ +This document describes how you can convert Google Code wiki pages to [GitHub-flavored Markdown](https://help.github.com/articles/github-flavored-markdown/). This variant of Markdown is also compatible with Bitbucket’s wiki format. + +Once the Google Code project's .wiki files have been converted to Markdown, you will need to import them into your GitHub or Bitbucket repo using their web UI, or check the files directly into the repo. + +# Install Python # +First you need to [install Python](https://www.python.org/downloads/), though it should be already installed on most Mac and Linux machines. + +Next, [clone the support-tools Google Code project](https://code.google.com/p/support-tools/source/checkout) (i.e. _this_ project's source code). + +``` +git clone https://code.google.com/p/support-tools/ +``` + +# Run the Converter # +With the conversion tool, all you need to do is simply run it on your project’s wiki pages. + +Google Code stores wikis along with project source code, but the exact location depends on the type of project (Subversion, Mercurial, or Git). + +Git and Mercurial projects store wikis in a separate repo, so to get access to your wiki files you need to clone the "wiki" subrepo. For example, if you want to get access to all of the wiki files in the "codesearch" project, simply clone the "codesearch.wiki" repository. + +``` +hg clone https://code.google.com/p/codesearch.wiki/ +``` + +Subversion projects store the wikis directly in the svn repository, in the `/wiki` folder. + +## Running the converter ## + +To run the `wiki2gfm` tool, simply run: + +``` +python ./wiki2gfm.py \ + --input_file \ + --output_file +``` + +The conversion process may output warnings to the console. This is expected as Google Code’s wiki syntax cannot always be converted to Markdown. Take a look over these warnings, along with the output to ensure no undesirable formatting has occurred. + +There are several additional flags that may be of use: + +`--project`: The name of your Google Code project. It is possible to refer to your project name indirectly in wiki syntax; providing this option makes it possible for the converter to resolve your project name when this occurs. + +`--wikipages_list`: The list of wiki pages that are assumed to exist for the purpose of auto-linking to other pages. Wiki syntax allows auto-linking to other wiki pages, if they exist. Providing a list of wiki pages that do actually exist to the converter allows it to optimistically link to the converted page of another wiki. + +`--wikipages_path`: An alternate way of providing the same information as --wikipages\_list, this is a list of directories to search for .wiki files, the names of which are used as if they were provided via --wikipages\_list. + +`--[no_]symmetric_headers`: Controls whether or not headers in Markdown are given symmetric header guards. In Markdown, a header is indicated by a number of # symbols, and only the leading group of symbols is required; however, for aesthetic purposes it is sometimes desirable for the header to be surrounded by equal numbers of #. This flag controls this. + +# Bulk conversion # +For projects with many wiki pages, manually typing in the name of each wiki file can get tedious. To bulk-convert all of a project’s wiki pages, you can use the following Bash script: + +``` +#!/bin/bash + +# Bulk converts wiki pages. Usage: +# bulk-convert.sh +USAGE="Bulk converter for wiki pages. + +Example usage: +$ mkdir converted-wikis +$ ./bulk-convert.sh \ + ./support-tools/wiki_to_md/wiki2gfm.py \ + ./google-code-project.wiki/ \ + ./converted-wikis/ +" + +if [ $# -eq 0 ] ; then + echo "$USAGE" + echo + exit 1 +fi + +PATH_TO_WIKI2GMF=$1 +PATH_TO_WIKIS=$2 +OUTPUT_DIR=$3 + +for file in `ls $PATH_TO_WIKIS` +do + printf "**************************\n" + printf "Converting $file\n" + printf "**************************\n" + + python $PATH_TO_WIKI2GMF \ + --input_file=$PATH_TO_WIKIS/$file \ + --output_file=$OUTPUT_DIR/$file.md + + printf "done\n\n" + +done +``` \ No newline at end of file