Skip to content
GitLab
Explore
Sign in
Register
Primary navigation
Search or go to…
Project
G
Greenlight
Manage
Activity
Members
Labels
Plan
Issues
Issue boards
Milestones
Wiki
Code
Merge requests
Repository
Branches
Commits
Tags
Repository graph
Compare revisions
Snippets
Build
Pipelines
Jobs
Pipeline schedules
Artifacts
Deploy
Releases
Package registry
Container registry
Model registry
Operate
Environments
Terraform modules
Monitor
Incidents
Analyze
Value stream analytics
Contributor analytics
CI/CD analytics
Repository analytics
Model experiments
Help
Help
Support
GitLab documentation
Compare GitLab plans
GitLab community forum
Contribute to GitLab
Provide feedback
Terms and privacy
Keyboard shortcuts
?
Snippets
Groups
Projects
Show more breadcrumbs
Arbeitsgruppe Hardwarenahe IT-Systeme
Greenlight
Commits
42373e20
Unverified
Commit
42373e20
authored
Oct 20, 2022
by
Khemissi Amir
Committed by
GitHub
Oct 20, 2022
Browse files
Options
Downloads
Patches
Plain Diff
Resources migration: Enabled migration tasks with batch processing. (#4010)
parent
779c15d9
No related branches found
No related tags found
No related merge requests found
Changes
1
Show whitespace changes
Inline
Side-by-side
Showing
1 changed file
lib/tasks/migrations/migrations.rake
+50
-33
50 additions, 33 deletions
lib/tasks/migrations/migrations.rake
with
50 additions
and
33 deletions
lib/tasks/migrations/migrations.rake
+
50
−
33
View file @
42373e20
...
@@ -2,14 +2,17 @@
...
@@ -2,14 +2,17 @@
namespace
:migrations
do
namespace
:migrations
do
COMMON
=
{
COMMON
=
{
headers:
{
"Content-Type"
=>
"application/json"
}
headers:
{
"Content-Type"
=>
"application/json"
},
batch_size:
1000
,
}.
freeze
}.
freeze
desc
"Migrates v2 resources to v3"
desc
"Migrates v2 resources to v3"
task
:roles
,
[]
=>
:environment
do
|
_task
,
_args
|
task
:roles
,
[]
=>
:environment
do
|
_task
,
_args
|
has_encountred_issue
=
0
has_encountred_issue
=
0
Role
.
select
(
:id
,
:name
).
where
.
not
(
name:
Role
::
RESERVED_ROLE_NAMES
).
each
do
|
r
|
Role
.
select
(
:id
,
:name
)
.
where
.
not
(
name:
Role
::
RESERVED_ROLE_NAMES
)
.
find_each
(
batch_size:
COMMON
[
:batch_size
])
do
|
r
|
params
=
{
role:
{
name:
r
.
name
}
}
params
=
{
role:
{
name:
r
.
name
}
}
response
=
Net
::
HTTP
.
post
(
uri
(
'roles'
),
payload
(
params
),
COMMON
[
:headers
])
response
=
Net
::
HTTP
.
post
(
uri
(
'roles'
),
payload
(
params
),
COMMON
[
:headers
])
...
@@ -17,11 +20,11 @@ namespace :migrations do
...
@@ -17,11 +20,11 @@ namespace :migrations do
when
Net
::
HTTPCreated
when
Net
::
HTTPCreated
puts
green
"Succesfully migrated Role:"
puts
green
"Succesfully migrated Role:"
puts
cyan
" ID:
#{
r
.
id
}
"
puts
cyan
" ID:
#{
r
.
id
}
"
puts
cyan
" Name:
#{
r
.
name
}
"
puts
cyan
" Name:
#{
params
[
:role
][
:
name
]
}
"
else
else
puts
red
"Unable to migrate Role:"
puts
red
"Unable to migrate Role:"
puts
yellow
" ID:
#{
r
.
id
}
"
puts
yellow
" ID:
#{
r
.
id
}
"
puts
yellow
" Name:
#{
r
.
name
}
"
puts
yellow
" Name:
#{
params
[
:role
][
:
name
]
}
"
has_encountred_issue
=
1
# At least one of the migrations failed.
has_encountred_issue
=
1
# At least one of the migrations failed.
end
end
end
end
...
@@ -32,11 +35,13 @@ namespace :migrations do
...
@@ -32,11 +35,13 @@ namespace :migrations do
exit
has_encountred_issue
exit
has_encountred_issue
end
end
task
:users
,
[]
=>
:environment
do
|
_task
,
_args
|
task
:users
,
[
:start
,
:stop
]
=>
:environment
do
|
_task
,
args
|
start
,
stop
=
range
(
args
)
has_encountred_issue
=
0
has_encountred_issue
=
0
# TODO: Optimize this by running in batches.
User
.
select
(
:id
,
:uid
,
:name
,
:email
,
:social_uid
,
:language
,
:role_id
)
User
.
select
(
:id
,
:uid
,
:name
,
:email
,
:social_uid
,
:language
,
:role_id
).
each
do
|
u
|
.
find_each
(
start:
start
,
finish:
stop
,
batch_size:
COMMON
[
:batch_size
])
do
|
u
|
params
=
{
user:
{
name:
u
.
name
,
email:
u
.
email
,
external_id:
u
.
social_uid
,
language:
u
.
language
,
role:
u
.
role
.
name
}
}
params
=
{
user:
{
name:
u
.
name
,
email:
u
.
email
,
external_id:
u
.
social_uid
,
language:
u
.
language
,
role:
u
.
role
.
name
}
}
response
=
Net
::
HTTP
.
post
(
uri
(
'users'
),
payload
(
params
),
COMMON
[
:headers
])
response
=
Net
::
HTTP
.
post
(
uri
(
'users'
),
payload
(
params
),
COMMON
[
:headers
])
...
@@ -94,4 +99,16 @@ namespace :migrations do
...
@@ -94,4 +99,16 @@ namespace :migrations do
res
=
{
"v2"
=>
{
"encrypted_params"
=>
encrypt_params
(
params
)
}
}
res
=
{
"v2"
=>
{
"encrypted_params"
=>
encrypt_params
(
params
)
}
}
res
.
to_json
res
.
to_json
end
end
def
range
(
args
)
start
=
args
[
:start
].
to_i
start
=
1
unless
start
.
positive?
stop
=
args
[
:stop
].
to_i
stop
=
nil
unless
stop
.
positive?
raise
red
"Unable to migrate: Invalid provided range [start:
#{
start
}
, finish:
#{
stop
}
]"
if
stop
&&
start
>
stop
[
start
,
stop
]
end
end
end
This diff is collapsed.
Click to expand it.
Preview
0%
Loading
Try again
or
attach a new file
.
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Save comment
Cancel
Please
register
or
sign in
to comment