Bpcg with direct solve #515
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* sparsifier active set * fix typo * added sparsifying tests * generic tolerane * remove sparsification * format * HiGHS dep
* sparsifier active set * start working on LP AS * first working quadratic * remove quadratic LP from current * cleanup * HiGHS in test deps * working reworked LP quadratic * working version generic quadratic * slow version generic quadratic * faster term manipulation * copy sufficient * remove comment * added test for quadratic * minor * simplify example * clean up code, verify error with ASQuad * Add update_weights! to fix direct solve with active_set_quadratic * remove direct solve from BPCG * rng changed --------- Co-authored-by: Sébastien Designolle <[email protected]>
|
reopened the branch from before |
pokutta
commented
Oct 25, 2024
| MOI.empty!(o) | ||
| λ = MOI.add_variables(o, nv) | ||
| # λ ≥ 0, ∑ λ == 1 | ||
| # MOI.add_constraint.(o, λ, MOI.GreaterThan(0.0)) |
There was a problem hiding this comment.
commented out the nonnegativity constraint
pokutta
commented
Oct 25, 2024
Comment on lines
292
to
324
| # Wolfe's pull back | ||
| ## Extract lambda values into a vector | ||
| lambda_values = Vector{Float64}(undef, length(λ)) | ||
| for idx in eachindex(λ) | ||
| lambda_values[idx] = MOI.get(o, MOI.VariablePrimal(), λ[idx]) | ||
| end | ||
|
|
||
| ## Load old weights into vector and do ratio test | ||
| old_weights = as.weights | ||
| tau_min = 1.0 | ||
| for i in 1:length(lambda_values) | ||
| if lambda_values[i] < old_weights[i] | ||
| tau_min = min(tau_min, old_weights[i] / (old_weights[i] - lambda_values[i])) | ||
| end | ||
| end | ||
| tau = tau_min | ||
|
|
||
| # println("lambda_values: ", lambda_values) | ||
| # println("tau: ", tau) | ||
|
|
||
| ## Compute new lambdas | ||
| new_lambdas = Vector{R}(undef, length(lambda_values)) | ||
| for i in 1:length(lambda_values) | ||
| new_lambdas[i] = (1 - tau) * old_weights[i] + tau * lambda_values[i] | ||
| end | ||
|
|
||
| @assert all(0 .<= new_lambdas .<= 1) "All new_lambdas must be between 0 and 1" | ||
| @assert isapprox(sum(new_lambdas), 1.0, atol=1e-5) "The sum of new_lambdas must be approximately 1" | ||
|
|
||
| indices_to_remove = Int[] | ||
| new_weights = R[] | ||
| for idx in eachindex(λ) | ||
| weight_value = new_lambdas[idx] # using new lambdas |
There was a problem hiding this comment.
this should implement wolfe's pullback. get the unconstrained solution. form the line segment between current solution and unconstrained solution. compute tau on the line so that you stay feasible. update lambda and use
pokutta
commented
Oct 25, 2024
Comment on lines
95
to
127
| active_set_quadratic_automatic_standard = FrankWolfe.ActiveSetQuadraticLinearSolve( | ||
| FrankWolfe.ActiveSet([(1.0, copy(x00))]), | ||
| grad!, | ||
| MOI.instantiate(MOI.OptimizerWithAttributes(HiGHS.Optimizer, MOI.Silent() => true)), | ||
| scheduler=FrankWolfe.LogScheduler(start_time=10, scaling_factor=1), | ||
| ) | ||
| trajectoryBPCG_quadratic_automatic_standard = [] | ||
| x, v, primal, dual_gap, _ = FrankWolfe.blended_pairwise_conditional_gradient( | ||
| f, | ||
| grad!, | ||
| lmo, | ||
| active_set_quadratic_automatic_standard, | ||
| max_iteration=k, | ||
| verbose=true, | ||
| callback=build_callback(trajectoryBPCG_quadratic_automatic_standard), | ||
| ); | ||
|
|
||
|
|
||
| active_set_quadratic_wolfe = FrankWolfe.ActiveSetQuadraticLinearSolveWolfe( | ||
| FrankWolfe.ActiveSet([(1.0, copy(x00))]), | ||
| grad!, | ||
| MOI.instantiate(MOI.OptimizerWithAttributes(HiGHS.Optimizer, MOI.Silent() => true)), | ||
| scheduler=FrankWolfe.LogScheduler(start_time=10, scaling_factor=1), | ||
| ) | ||
| trajectoryBPCG_quadratic_wolfe = [] | ||
| x, v, primal, dual_gap, _ = FrankWolfe.blended_pairwise_conditional_gradient( | ||
| f, | ||
| grad!, | ||
| lmo, | ||
| active_set_quadratic_wolfe, | ||
| max_iteration=k, | ||
| verbose=true, | ||
| callback=build_callback(trajectoryBPCG_quadratic_wolfe), |
There was a problem hiding this comment.
these two runs cycle (irrespective of using the Wolfe one or the non-Wolfe one)
|
still some minor issue - run the direct_solve example with seed 10 @matbesancon maybe it is just numerics or some edge case. we need to check also for the Lp-norm LMO: |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.