Problem running automate-git.py for 2171 release branch

[Hylinn]

When running automate-git.py such that the command is

Code: Select all

python ..\depot_tools\cef3_2171_automate\automate-git.py --download-dir=C:\Users\htaggart\workspace\cef3_2171 --branch=2171 --force-clean --no-build

I keep getting the same error after fetching src/third_party/WebKit.

The console output is as follows:

Code: Select all

[0:45:07] Still working on:
[0:45:07]   src/third_party/WebKit
Syncing projects: 100% (92/92), done.
-------- Running "C:\Users\htaggart\workspace\cef3_2171\depot_tools\git.bat config --get remote.origin.url" in "C:\Users\htaggart\workspace\cef3_2171\chromium\src"...
--> Chromium URL: https://chromium.googlesource.com/chromium/src.git
-------- Running "C:\Users\htaggart\workspace\cef3_2171\depot_tools\git.bat rev-parse HEAD" in "C:\Users\htaggart\workspace\cef3_2171\chromium\src"...
-------- Running "C:\Users\htaggart\workspace\cef3_2171\depot_tools\git.bat rev-parse refs/tags/39.0.2171.95" in "C:\Users\htaggart\workspace\cef3_2171\chromium\src"...
--> Chromium Current Checkout: e5213fce048c62b058e9c3c9b9906cc2d0af182b
-------- Running "C:\Users\htaggart\workspace\cef3_2171\depot_tools\git.bat log --grep=^git-svn-id: -n 1 e5213fce048c62b058e9c3c9b9906cc2d0af182b" in "C:\Users\htaggart\workspace\cef3_2171\chromium\src"...
--> Chromium Current Revision: 292840
--> Chromium Desired Checkout: 07d25682862707e7524037c8b93ef93f361835f1 (refs/tags/39.0.2171.95)
-------- Running "C:\Users\htaggart\workspace\cef3_2171\depot_tools\git.bat log --grep=^git-svn-id: -n 1 07d25682862707e7524037c8b93ef93f361835f1" in "C:\Users\htaggart\workspace\cef3_2171\chromium\src"...
--> Chromium Desired Revision: 292145
-------- Running "C:\Users\htaggart\workspace\cef3_2171\depot_tools\git.bat fetch" in "C:\Users\htaggart\workspace\cef3_2171\chromium\src"...
remote: Counting objects: 107256, done
remote: Finding sources: 100% (115/115)
remote: Total 115 (delta 101), reused 115 (delta 101)
Receiving objects: 100% (115/115), 26.38 KiB | 0 bytes/s, done.
Resolving deltas: 100% (101/101), completed with 89 local objects.
From https://chromium.googlesource.com/chromium/src
   e5213fc..c2bf910  git-svn    -> origin/git-svn
   e5213fc..c2bf910  master     -> origin/master
   c3ef7fc..70b5863  refs/branch-heads/2272 -> branch-heads/2272
-------- Running "C:\Users\htaggart\workspace\cef3_2171\depot_tools\git.bat fetch --tags" in "C:\Users\htaggart\workspace\cef3_2171\chromium\src"...
-------- Running "C:\Users\htaggart\workspace\cef3_2171\depot_tools\git.bat checkout --force refs/tags/39.0.2171.95" in "C:\Users\htaggart\workspace\cef3_2171\chromium\src"...
error: unable to create file base/process/launch_win.cc (Permission denied)
error: unable to create file chrome/browser/extensions/extension_action_test_util.cc (Permission denied)
error: unable to create file chrome/browser/prefs/pref_service_syncable_factory.h (Permission denied)
error: unable to create file components/translate_strings.grdp (Permission denied)
error: unable to create file third_party/polymer/components-chromium/paper-progress/demo.html (Permission denied)
error: unable to create file third_party/polymer/components/core-icon/demo.html (Permission denied)
error: unable to create file tools/clang/blink_gc_plugin/tests/class_requires_trace_method.h (Permission denied)
Checking out files: 100% (41137/41137), done.
Previous HEAD position was e5213fc... Modify SetAlternateProtocol logging behavior.
HEAD is now at 07d2568... Publish DEPS for Chromium 39.0.2171.95
Traceback (most recent call last):
  File "..\depot_tools\cef3_2171_automate\automate-git.py", line 830, in <module>
    chromium_src_dir, depot_tools_dir)
  File "..\depot_tools\cef3_2171_automate\automate-git.py", line 74, in run
    shell=(sys.platform == 'win32'))
  File "C:\Users\htaggart\depot_tools\python276_bin\lib\subprocess.py", line 540, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['C:\\Users\\htaggart\\workspace\\cef3_2171\\depot_tools\\git.bat', 'checkout', '--force', 'refs/tags/39.0.2171.95']' returned non-zero exit status 1

I'm running the command prompt as an administrator and made sure that C:\Users\htaggart\workspace\cef3_2171 has its write privileges correctly set.

I did notice that the 2272 branch is mentioned in the output and thought that I might have an incorrect version of automate-git.py, but couldn't find any explicit reference to 2272 in the version that I have.

Any guidance on this issue would be greatly appreciated.

[magreenblatt]

It could be some permissions issue related to downloading in a user's directory when not using that user's account. Try a clean checkout to a non-user directory. Also, it shouldn't be necessary to run as Administrator.

[Hylinn]

Thanks for the response. I tried changing dowload-dir of automate-git.py to C:\cef3_2171, but got practically the same issue:

Code: Select all

Syncing projects: 100% (92/92), done.
-------- Running "C:\cef3_2171\depot_tools\git.bat config --get remote.origin.url" in "C:\cef3_2171\chromium\src"...
--> Chromium URL: https://chromium.googlesource.com/chromium/src.git
-------- Running "C:\cef3_2171\depot_tools\git.bat rev-parse HEAD" in "C:\cef3_2171\chromium\src"...
-------- Running "C:\cef3_2171\depot_tools\git.bat rev-parse refs/tags/39.0.2171.95" in "C:\cef3_2171\chromium\src"...
--> Chromium Current Checkout: de1df47e5aef43721d79a49d9fa554355ecf87cc
-------- Running "C:\cef3_2171\depot_tools\git.bat log --grep=^git-svn-id: -n 1 de1df47e5aef43721d79a49d9fa554355ecf87cc" in "C:\cef3_2171\chromium\src"...
--> Chromium Current Revision: 292840
--> Chromium Desired Checkout: 07d25682862707e7524037c8b93ef93f361835f1 (refs/tags/39.0.2171.95)
-------- Running "C:\cef3_2171\depot_tools\git.bat log --grep=^git-svn-id: -n 1 07d25682862707e7524037c8b93ef93f361835f1" in "C:\cef3_2171\chromium\src"...
--> Chromium Desired Revision: 292145
-------- Running "C:\cef3_2171\depot_tools\git.bat fetch" in "C:\cef3_2171\chromium\src"...
remote: Counting objects: 110358, done
remote: Finding sources: 100% (119/119)
remote: Total 119 (delta 98), reused 119 (delta 98)
Receiving objects: 100% (119/119), 26.37 KiB | 0 bytes/s, done.
Resolving deltas: 100% (98/98), completed with 85 local objects.
From https://chromium.googlesource.com/chromium/src
   de1df47..c431c0a  git-svn    -> origin/git-svn
   02ea111..a34f618  ignore/foo -> origin/ignore/foo
   d495728..76a4e28  lkcr       -> origin/lkcr
   de1df47..c431c0a  master     -> origin/master
   9f5621a..872df78  refs/branch-heads/2288 -> branch-heads/2288
 * [new tag]         42.0.2285.3 -> 42.0.2285.3
 * [new tag]         42.0.2288.6 -> 42.0.2288.6
-------- Running "C:\cef3_2171\depot_tools\git.bat fetch --tags" in "C:\cef3_2171\chromium\src"...
-------- Running "C:\cef3_2171\depot_tools\git.bat checkout --force refs/tags/39.0.2171.95" in "C:\cef3_2171\chromium\src"...
error: unable to create file ash/touch/touch_hud_debug.cc (Permission denied)
error: unable to create file base/prefs/writeable_pref_store.h (Permission denied)
error: unable to create file cc/test/fake_layer_tree_host_client.cc (Permission denied)
error: unable to create file chrome/browser/apps/ephemeral_app_browsertest.h (Permission denied)
error: unable to create file chrome/browser/chromeos/policy/enterprise_install_attributes.cc (Permission denied)
error: unable to create file chrome/browser/media_galleries/fileapi/media_file_validator_browsertest.cc (Permission denied)
error: unable to create file chrome/browser/resources/user_manager/user_manager_tutorial.html (Permission denied)
error: unable to create file chrome/browser/ui/cocoa/infobars/infobar_container_cocoa.h (Permission denied)
error: unable to create file chrome/browser/ui/cocoa/validation_message_bubble_cocoa.mm (Permission denied)
error: unable to create file chrome/browser/ui/webui/bookmarks_ui_browsertest.cc (Permission denied)
error: unable to create file chrome/common/extensions/api/gcd_private.idl (Permission denied)
error: unable to create file components/cdm/common/OWNERS (Permission denied)
error: unable to create file content/browser/download/download_request_handle.h (Permission denied)
error: unable to create file content/browser/geolocation/network_location_provider.h (Permission denied)
error: unable to create file extensions/common/api/api.gyp (Permission denied)
error: unable to create file gin/public/debug.h (Permission denied)
error: unable to create file gpu/command_buffer/service/context_state.h (Permission denied)
error: unable to create file media/base/simd/convert_yuv_to_rgb_c.cc (Permission denied)
error: unable to create file media/formats/mp4/avc.h (Permission denied)
error: unable to create file native_client_sdk/doc_generated/pepper_beta/cpp/classpp_1_1_touch_input_event__inherit__graph.png (Permission denied)
error: unable to create file native_client_sdk/src/libraries/nacl_io/kernel_wrap_glibc.cc (Permission denied)
error: unable to create file net/cert/x509_certificate_nss.cc (Permission denied)
error: unable to create file net/quic/quic_utils_chromium_test.cc (Permission denied)
error: unable to create file net/quic/test_tools/crypto_test_utils_chromium.cc (Permission denied)
error: unable to create file remoting/webapp/js_proto/dom_proto.js (Permission denied)
error: unable to create file third_party/polymer/components/core-header-panel/bower.json (Permission denied)
error: unable to create file third_party/polymer/components/core-scroll-header-panel/demos/demo3.html (Permission denied)
error: unable to create file ui/events/event_processor.cc (Permission denied)
Checking out files: 100% (41275/41275), done.
Previous HEAD position was de1df47... Re-prompt Chrome users to run the SRT when a finch param changes.
HEAD is now at 07d2568... Publish DEPS for Chromium 39.0.2171.95
Traceback (most recent call last):
  File "..\depot_tools\cef3_2171_automate\automate-git.py", line 830, in <module>
    chromium_src_dir, depot_tools_dir)
  File "..\depot_tools\cef3_2171_automate\automate-git.py", line 74, in run
    shell=(sys.platform == 'win32'))
  File "C:\Users\htaggart\depot_tools\python276_bin\lib\subprocess.py", line 540, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['C:\\cef3_2171\\depot_tools\\git.bat', 'checkout', '--force', 'refs/tags/39.0.2171.95']' returned non-zero exit status 1

I find it interesting that the files are more numerous the second time around. I'm going to dig a bit deeper to see if this is indeed a local permissions issue, since I'm now unable to properly download 2171 to two separate locations. Until then any help or pointers are welcome.

[gaetano]

I am running into the same issue with branch 2171 - something seems to be broken

[csmithizo]

I am also having problems, trying to build on OSX 10.10, Python 2.7

Running command "python ../automate-git.py --download-dir=. --branch=2171"

after downloading CEF and Chromium, it is failing with this output

Code: Select all

-------- Running "/Users/csmith/cef_2ndtry/chromium/src/cef/cef_create_projects.sh" in "/Users/csmith/cef_2ndtry/chromium/src/cef"...

Checking CEF and Chromium revisions...

________ running '/usr/bin/python tools/check_revision.py' in '/Users/csmith/cef_2ndtry/chromium/src/cef'
Traceback (most recent call last):
  File "tools/check_revision.py", line 7, in <module>
    from svn_util import *
ImportError: No module named svn_util
Traceback (most recent call last):
  File "tools/gclient_hook.py", line 16, in <module>
    RunAction(cef_dir, gyper)
  File "/Users/csmith/cef_2ndtry/chromium/src/cef/tools/gclient_util.py", line 39, in RunAction
    command, cwd=dir, always=True)
  File "/Users/csmith/cef_2ndtry/depot_tools/gclient_utils.py", line 293, in CheckCallAndFilterAndHeader
    return CheckCallAndFilter(args, **kwargs)
  File "/Users/csmith/cef_2ndtry/depot_tools/gclient_utils.py", line 538, in CheckCallAndFilter
    rv, args, kwargs.get('cwd', None), None, None)
subprocess2.CalledProcessError: Command /usr/bin/python tools/check_revision.py returned non-zero exit status 1 in /Users/csmith/cef_2ndtry/chromium/src/cef

This seems to be trying to read an svn file to get the Chromium version (and I have no svn Python library), but there is no svn file to read with the Chromium that was download, nor would I expect there to be since they are now using a git workflow, correct? Even if I find that svn python library there is nothing for it to read.

I tried doing the manual download steps, but gclient_hook.py still seems to fail trying to verify the Chromium version.

Before I break this down further and try to do everything by hand, can someone tell me what I might be doing wrong, or is there perhaps a problem with this version of automat-git.py?

Thanks!
Carlos Smith
iZotope, Inc.

[magreenblatt]

Does the file src/cef/tools/svn_util.py exist? Try using an absolute path for download_dir.

[csmithizo]

Hi Marshall,

No, there is no cef/tools/svn_util.py downloaded.

Just to make sure there is no discrepancy, I downloaded automate-git.py from the link on this page: https://bitbucket.org/chromiumembedded/ ... ndBuilding, so the target was https://bitbucket.org/chromiumembedded/ ... ate-git.py

I will try running it again with an absolute directory for the download-dir argument.

Thanks,
Carlos Smith

[csmithizo]

Unfortunately I am still getting the same result.

I am running the command (with absolute path)

Code: Select all

python automate-git.py --branch=2171 --download-dir=/users/csmith/cef_download/cef_automated


This is the output in the area of the error.

Code: Select all

--> GYP_GENERATORS=ninja
-------- Running "/users/csmith/cef_download/cef_automated/chromium/src/cef/cef_create_projects.sh" in "/users/csmith/cef_download/cef_automated/chromium/src/cef"...

Checking CEF and Chromium revisions...

________ running '/usr/bin/python tools/check_revision.py' in '/Users/csmith/CEF_download/cef_automated/chromium/src/cef'
Traceback (most recent call last):
  File "tools/check_revision.py", line 7, in <module>
    from svn_util import *
ImportError: No module named svn_util
Traceback (most recent call last):
  File "tools/gclient_hook.py", line 16, in <module>
    RunAction(cef_dir, gyper)
  File "/Users/csmith/CEF_download/cef_automated/chromium/src/cef/tools/gclient_util.py", line 39, in RunAction
    command, cwd=dir, always=True)
  File "/users/csmith/cef_download/cef_automated/depot_tools/gclient_utils.py", line 293, in CheckCallAndFilterAndHeader
    return CheckCallAndFilter(args, **kwargs)
  File "/users/csmith/cef_download/cef_automated/depot_tools/gclient_utils.py", line 538, in CheckCallAndFilter
    rv, args, kwargs.get('cwd', None), None, None)
subprocess2.CalledProcessError: Command /usr/bin/python tools/check_revision.py returned non-zero exit status 1 in /Users/csmith/CEF_download/cef_automated/chromium/src/cef
Traceback (most recent call last):
  File "automate-git.py", line 815, in <module>
    run(path, cef_src_dir, depot_tools_dir)
  File "automate-git.py", line 55, in run
    shell=(sys.platform == 'win32'))
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 540, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/users/csmith/cef_download/cef_automated/chromium/src/cef/cef_create_projects.sh']' returned non-zero exit status 1
csmith-mac:cef_download csmith$ python automate-git.py --branch=2171 --download-dir=/users/csmith/cef_download/cef_automated

I think the real problem here is that the version of check_revision.py in ~/CEF_download/cef_automated/chromium/src/cef is looking for svn.

The code in check_revision.py I am seeing says (starting at line 31)

Code: Select all

if not os.path.exists(os.path.join(cef_dir, '.svn')) or \
   not os.path.exists(os.path.join(src_dir, '.svn')):
  sys.stdout.write("Not an SVN checkout.\n")
  sys.exit(0)

# Retrieve the CEF SVN info.
cef_info = get_svn_info(cef_dir)

There is no .svn file. There is a .git directory and other associated git files and directories.

It appears to me that there is a mismatch between the CEF that automate-git.py is bringing in, and the version of CEF included in the Chromium branch.

There is a chromium/src/chrome/VERSION file, it contains
MAJOR=39
MINOR=0
BUILD=2171
PATCH=95

Thanks for any help,
Carlos Smith
iZotope, Inc.

[magreenblatt]

The check_revision.py problem has been fixed in https://bitbucket.org/chromiumembedded/ ... c1?at=2171.

[csmithizo]

The check_revision.py problem has been fixed in https://bitbucket.org/chromiumembedded/ ... c1?at=2171.

Thank you sir, that seems to have done the trick!

Carlos Smith
iZotope, Inc.