Building old CEF 5414 failed : gclient 'returned non-zero'

[RCK]

Hello guys,

I'm trying to build CEF 5414, so I read :
https://bitbucket.org/chromiumembedded/ ... ndBuilding
https://bitbucket.org/chromiumembedded/ ... ldSetup.md
https://chromium.googlesource.com/chrom ... visions.md

And proceed like this :
- Fresh Windows 10 LTSC 21H2 on 500GB SSD + 32 GB ram + Defender and Windows search stopped.
- Latest Visual Studio 2019 Enterprise (16.11.32) with C++ desktop dev + ATL + MFC + SDK 10.0.20348.
- Folder created

c:\code
c:\code\automate
c:\code\chromium_git

- Added "c:\code\depot_tools" at the beginning of the "Path" environnement system variable
- Depot_Tools rolled back to 2023-01-26 00:00:00 +0000

latest depot_tools installed from https://storage.googleapis.com/chrome-i ... _tools.zip
updated with gclient to have git
new environnement system variable: DEPOT_TOOLS_UPDATE=0
cd c:\code\depot_tools
git rev-list -n 1 --before="2023-01-26 00:00:00 +0000" main = 44e9bee34fcc7e8dedc2a988750c9d7cc7f73eba
git checkout 44e9bee34fcc7e8dedc2a988750c9d7cc7f73eba
git clean -ffd

- automate-git with version 5414 from https://bitbucket.org/chromiumembedded/ ... ate-git.py
- then start compilation with

set GN_DEFINES=is_official_build=true
set GYP_MSVS_VERSION=2019
set CEF_ARCHIVE_FORMAT=tar.bz2
python3 c:\code\automate\automate-git.py --download-dir=c:\code\chromium_git --depot-tools-dir=c:\code\depot_tools --branch=5414 --no-depot-tools-update --x64-build --client-distrib-only

Unfortunatly, it failed with the following error (full log attached if needed)

Traceback (most recent call last):
File "c:\code\automate\automate-git.py", line 1299, in <module>
run("gclient sync --nohooks --with_branch_heads --jobs 16", \
File "c:\code\automate\automate-git.py", line 69, in run
return subprocess.check_call(
File "C:\code\depot_tools\bootstrap-2@3_8_10_chromium_26_bin\python3\bin\lib\subprocess.py", line 364, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['gclient', 'sync', '--nohooks', '--with_branch_heads', '--jobs', '16']' returned non-zero exit status 1.
C:\code\depot_tools>

Did someone have one idea to fix this ?
Thanks !

cef_5414.txt

build error for CEF 5414

(718.21 KiB) Downloaded 242 times

[RCK]

Just to be sure, I also tested manual build with the following C:\code\chromium_git\update.bat

set GN_DEFINES=is_component_build=true
set GN_ARGUMENTS=--ide=vs2019 --sln=cef --filters=//cef/*
python3 ..\automate\automate-git.py --download-dir=c:\code\chromium_git --depot-tools-dir=c:\code\depot_tools --no-distrib --no-build --no-depot-tools-update --branch=5414

It produce the same problem:

.../assistant/internal', 'src/components/optimization_guide/internal', 'src/ios_internal']}
Traceback (most recent call last):
File "..\automate\automate-git.py", line 1299, in <module>
run("gclient sync --nohooks --with_branch_heads --jobs 16", \
File "..\automate\automate-git.py", line 69, in run
return subprocess.check_call(
File "C:\code\depot_tools\bootstrap-2@3_8_10_chromium_26_bin\python3\bin\lib\subprocess.py", line 364, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['gclient', 'sync', '--nohooks', '--with_branch_heads', '--jobs', '16']' returned non-zero exit status 1.
C:\code\chromium_git>

What can I do to find the problem ?

[ndesktop]

My 5414 build's depot_tools is on

Code: Select all

SHA-1: d9e2d47985743491d3aa3f74398ada910c9e4fe2

* Roll recipe dependencies (trivial).

This is an automated CL created by the recipe roller. This CL rolls
recipe changes from upstream projects (recipe_engine) into this repository.

The build that created this CL was
https://ci.chromium.org/b/8786957573508306657

recipe_engine:
https://chromium.googlesource.com/infra/luci/recipes-py.git/+/b12b29b1516133836ef7fbd61f43bb7202efeb7e
  b12b29b (chromium-autoroll@skia-public.iam.gserviceaccount.com)
      Roll CAS Client from 0a1b43bf5869 to 320bf3ed60cd

Please check the following references for more information:
- autoroller, https://chromium.googlesource.com/infra/luci/recipes-py/+/main/doc/workflow.md#autoroller
- rollback, https://chromium.googlesource.com/infra/luci/recipes-py/+/main/doc/workflow.md#rollback
- cross-repo dependencies, https://chromium.googlesource.com/infra/luci/recipes-py/+/main/doc/cross_repo.md

Use https://goo.gl/noib3a to file a bug.

R=iannucci@chromium.org

Recipe-Tryjob-Bypass-Reason: Autoroller
Ignore-Freeze: Autoroller
Bugdroid-Send-Email: False
Change-Id: I489058a662b9eae6b20fc1145f74435bd4d4a2a8
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4330602
Commit-Queue: Recipe Roller <recipe-mega-autoroller@chops-service-accounts.iam.gserviceaccount.com>
Bot-Commit: Recipe Roller <recipe-mega-autoroller@chops-service-accounts.iam.gserviceaccount.com>


Your commit seems to be

Code: Select all

SHA-1: 44e9bee34fcc7e8dedc2a988750c9d7cc7f73eba

* Allow --force to be used in cl patch with --reapply
Bug: b/265929888
Change-Id: I7ed00d70d18463b3394915ae4c74f853d05cb88d
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4191996
Commit-Queue: Joanna Wang <jojwang@chromium.org>
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>

Maybe this helps.

Note. My build script did *not* update depot_tools (at 5414 time, update script called goma_ctl.bat which did not worked).

[RCK]

Hello ndesktop, thanks for your reply

So I tried this:

- delete C:\code\depot_tools
- set DEPOT_TOOLS_UPDATE=0
- unzip https://storage.googleapis.com/chrome-i ... _tools.zip into C:\code\depot_tools
- install git and python with C:\code\depot_tools\bootstrap>win_tools.bat
- check current commit with "git log -n 1" : commit 25b4354fadce5fc7d318e18bef43e6dc8e68e10d (HEAD, origin/main, origin/HEAD, main) Author: Josip Sokcevic <sokcevic@chromium.org> Date: Thu Nov 23 02:54:15 2023 +0000
- downgrade to your version with "git checkout d9e2d47985743491d3aa3f74398ada910c9e4fe2" : Previous HEAD position was 25b4354f Revert "pylint: Add pylint 2.13" HEAD is now at d9e2d479 Roll recipe dependencies (trivial).
- check current commit with "git log -n 1" : commit d9e2d47985743491d3aa3f74398ada910c9e4fe2 (HEAD) Author: recipe-roller <recipe-mega-autoroller@chops-service-accounts.iam.gserviceaccount.com> Date: Sat Mar 11 02:26:48 2023 +0000 Roll recipe dependencies (trivial).

Redownload automate-git.py from 5414 cef version from https://bitbucket.org/chromiumembedded/ ... ate-git.py

Then relaunched the build --no-depot-tools-update with:

set GN_DEFINES=is_official_build=true
set GYP_MSVS_VERSION=2019
set CEF_ARCHIVE_FORMAT=tar.bz2
python3 c:\code\automate\automate-git.py --download-dir=c:\code\chromium_git --depot-tools-dir=c:\code\depot_tools --branch=5414 --no-depot-tools-update --x64-build --client-distrib-only

Without success

'src/ios_internal']}
Traceback (most recent call last):
File "c:\code\automate\automate-git.py", line 1376, in <module>
run("gclient revert --nohooks", chromium_dir, depot_tools_dir)
File "c:\code\automate\automate-git.py", line 69, in run
return subprocess.check_call(
File "C:\code\depot_tools\bootstrap-2@3_8_10_chromium_26_bin\python3\bin\lib\subprocess.py", line 364, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['gclient', 'revert', '--nohooks']' returned non-zero exit status 1.

Could you check on your side ?
- the SHA1 of automate-git.py = 1d7ce0917722d893eab97a328c571de622bc4869
- the current checkout of "cef" = f1c41e4b1392ef8816171c4d51a2fca308db5501 (Thu Jan 26 13:50:30 2023 -0500)

Maybe you have one different automate-git.py to share with me

[ndesktop]

I don't have automate-git.py retrieved via git, but downloaded whatever version was at that time. But automate-git.py usage is recommended to be used on the latest version, disregarding the CEF branch used.
Anyways, I am attaching the 5414 local version I have. Maybe it helps.
The CEF version is f1c41e4b1392ef8816171c4d51a2fca308db5501, same as yours.

[RCK]

Hi ndesktop,

Thanks for your reply
So, we are sure I have the same version than you for :
C:\code\automate\automate-git.py
C:\code\depot_tools\*

So I deleted C:\code\chromium_git and retry a full build with

set GN_DEFINES=is_official_build=true
set GYP_MSVS_VERSION=2019
set CEF_ARCHIVE_FORMAT=tar.bz2
python3 c:\code\automate\automate-git.py --download-dir=c:\code\chromium_git --depot-tools-dir=c:\code\depot_tools --branch=5414 --no-depot-tools-update --x64-build --client-distrib-only

Without success

cef_5414_2.txt

full log

(861.3 KiB) Downloaded 205 times

Some errors:

Error: 1> Wrong keys 'git_dependencies' in
...
File "C:\code\depot_tools\bootstrap-2@3_8_10_chromium_26_bin\python3\bin\lib\subprocess.py", line 364, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['gclient', 'sync', '--nohooks', '--with_branch_heads', '--jobs', '16']' returned non-zero exit status 1.

So, to be sure I try to only retreive sources

set GN_DEFINES=is_official_build=true
set GYP_MSVS_VERSION=2019
set CEF_ARCHIVE_FORMAT=tar.bz2
python3 c:\code\automate\automate-git.py --download-dir=c:\code\chromium_git --depot-tools-dir=c:\code\depot_tools --branch=5414 --no-depot-tools-update --no-build

And I got the same error:

cef_5414_2.txt

full log

(861.3 KiB) Downloaded 205 times

So I supposed there is something wrong in my Windows.
I plan to rebuild a dedicated PC to rebuild CEF, may I ask you:
- Can you try my "--no-build" command and tell me if it still working ?
- What version of windows are you using ?
- Do you have desactivated services like Win Defender or Win Search ?

Thanks

[ndesktop]

I am by no means an expert in gclient, but I am reading this error in another way.
I think something in DEPS (or another form of git dependency) does not exist anymore, or the requested revision.
From the attached log I don't see a possible culprit singled out, just the whole dependency.
Maybe there is a some --verbose flag of gclient or automate-git.py that would display some more info/warning that might point out what package is missing or not found?

[RCK]

You were right, I setup a new PC with Win10x64 + Defender disabled + Search disabled, and I got the same problem, no way to correctly sync Chromium 5414.

Calling the following command line download 55 GB and stop on the same error:

set GN_DEFINES=is_component_build=true
set GN_ARGUMENTS=--ide=vs2019 --sln=cef --filters=//cef/*
python3 c:\code\automate\automate-git.py --download-dir=c:\code\chromium_git --depot-tools-dir=c:\code\depot_tools --branch=5414 --no-depot-tools-update --no-distrib --no-build

When looking into automate-git.py, I feel the gclient call that is causing problem is this one:

# Initial Chromium checkout.
if not options.nochromiumupdate and not os.path.exists(chromium_src_dir):
chromium_checkout_new = True
run("gclient sync --nohooks --with_branch_heads --jobs 16", \
chromium_dir, depot_tools_dir)

So here is my .gclient conf

solutions = [{'managed': False,'name': 'src', 'url': 'https://chromium.googlesource.com/chromium/src.git', 'custom_vars': {'checkout_pgo_profiles': False, }, 'custom_deps': {'build': None, 'build/scripts/command_wrapper/bin': None, 'build/scripts/gsd_generate_index': None, 'build/scripts/private/data/reliability': None, 'build/scripts/tools/deps2git': None, 'build/third_party/lighttpd': None, 'commit-queue': None, 'depot_tools': None, 'src/chrome_frame/tools/test/reference_build/chrome': None, 'src/chrome/tools/test/reference_build/chrome_linux': None, 'src/chrome/tools/test/reference_build/chrome_mac': None, 'src/chrome/tools/test/reference_build/chrome_win': None, }, 'deps_file': 'DEPS', 'safesync_url': ''}]

And when I manually launch gclient --verbose , I have the following error:

Microsoft Windows [version 10.0.19044.3570]
(c) Microsoft Corporation. Tous droits réservés.

C:\Users\Administrateur\Desktop>cd c:\code\chromium_git\chromium

c:\code\chromium_git\chromium>gclient sync --nohooks --with_branch_heads --jobs 16 --verbose
Looking for .gclient starting from c:\code\chromium_git\chromium

Loaded .gclient config in c:\code\chromium_git\chromium:
solutions = [{'managed': False,'name': 'src', 'url': 'https://chromium.googlesource.com/chromium/src.git', 'custom_vars': {'checkout_pgo_profiles': False, }, 'custom_deps': {'build': None, 'build/scripts/command_wrapper/bin': None, 'build/scripts/gsd_generate_index': None, 'build/scripts/private/data/reliability': None, 'build/scripts/tools/deps2git': None, 'build/third_party/lighttpd': None, 'commit-queue': None, 'depot_tools': None, 'src/chrome_frame/tools/test/reference_build/chrome': None, 'src/chrome/tools/test/reference_build/chrome_linux': None, 'src/chrome/tools/test/reference_build/chrome_mac': None, 'src/chrome/tools/test/reference_build/chrome_win': None, }, 'deps_file': 'DEPS', 'safesync_url': ''}]

src
----------------------------------------
[0:00:00] Started.
_____ src at refs/remotes/origin/main
[0:00:00] ________ unmanaged solution; skipping src
[0:00:00] Finished running: git rev-parse --verify HEAD
[0:00:00] Finished running: git rev-parse HEAD
----------------------------------------

src (ERROR)
----------------------------------------
[0:00:00] Started.
_____ src at refs/remotes/origin/main
[0:00:00] ________ unmanaged solution; skipping src
[0:00:00] Finished running: git rev-parse --verify HEAD
[0:00:00] Finished running: git rev-parse HEAD
----------------------------------------
Error: 1> Wrong keys 'git_dependencies' in {'vars': {'build_with_chromium': True, 'checkout_configuration': 'default', 'checkout_android': False, 'checkout_fuchsia': False, 'checkout_fuchsia_internal': False, 'checkout_fuchsia_internal_images': '',...

I'm stuck with gclient not wanting me having Chrome 5414

[RCK]

@magreenblatt : any idea about gclient ?

[magreenblatt]

Error: 1> Wrong keys 'git_dependencies'

See viewtopic.php?f=6&t=19525#p53955