Bug 9395 - Cleanly support unicode throughout Toaster
Summary: Cleanly support unicode throughout Toaster
Status: RESOLVED WORKSFORME
Alias: None
Product: Toaster
Classification: Build System, Metadata & Runtime
Component: toaster (show other bugs)
Version: 2.2
Hardware: x86 x86
: Medium critical
Target Milestone: Future
Assignee: David Reyna
QA Contact: Libertad
URL:
Whiteboard:
Depends on:
Blocks:
 
Reported: 2016-04-05 12:25 UTC by Joshua Lock
Modified: 2017-06-15 15:10 UTC (History)
6 users (show)

See Also:
OS type for building Yocto: ---
Type of Regression: ---
Verified:
Documentation change: No (bug/feature does not impact docs)


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Joshua Lock 2016-04-05 12:25:52 UTC
In https://bugzilla.yoctoproject.org/show_bug.cgi?id=9142 we observed exceptions being raised when an ImagePkgList MetadataEvent contains non-ASCII characters. Worse still, these exceptions were invisible to the Toaster UI.

A workaround patch has been proposed for 2.1 to workaround the specific issue of handling file paths with non-ASCII characters, however we should make an effort in the 2.2 timeframe to ensure that we have unicode support throughout Toaster.

This is likely to be made much easier should the plan to switch BitBake to Python3 go ahead, see:

https://docs.python.org/2/howto/unicode.html
https://docs.python.org/3/howto/unicode.html
Comment 1 aveselinov 2016-10-19 07:47:00 UTC
I'm getting this error whenever I try to build:

'ascii' codec can't decode byte 0xc2 in position 10: ordinal not in range(128)

On Ubuntu 15.04 machine.
Comment 2 Joshua Lock 2016-10-19 15:08:32 UTC
(In reply to comment #1)
> I'm getting this error whenever I try to build:
> 
> 'ascii' codec can't decode byte 0xc2 in position 10: ordinal not in
> range(128)
> 
> On Ubuntu 15.04 machine.

Can you provide more details on your setup so that we can try and reproduce? Is there a more detailed backtrace available?
Comment 3 brian avery 2016-10-19 16:02:54 UTC
In particular, for additional information we need: 
toaster version (jethro,krogoth,morty,master...?)
build target : (core-image-minimal,bash,...?)
Additional Layers if any?
Variables changed if any?
Comment 4 aveselinov 2016-10-20 06:55:37 UTC
I have downloaded the vendor BSP for my board (which have a collection of meta layers + poky)
I have installed Toaster krogoth following the manual from yocto website.

compulab-bsp/
├── build
│   ├── buildhistory
│   ├── cache
│   ├── conf
│   └── tmp
├── downloads
│   ├── git2
│   └── uninative
├── sources
│   ├── base
│   ├── Documentation
│   ├── meta-browser      //not used
│   ├── meta-compulab      
│   ├── meta-fsl-arm
│   ├── meta-fsl-arm-extra
│   ├── meta-fsl-demos    //not used
│   ├── meta-openembedded
│   ├── meta-qt5          //not used
│   └── poky
├── _toaster_clones
└── venv
    ├── bin
    ├── include
    ├── lib
    └── local

I init Virtual env:
$ source venv/bin/activate

And start toaster:
$ source sources/poky/bitbake/bin/toaster

The webserver address that I get it isn't 127.0.0.1:8000 but 0.0.0.0:8000

target image: core-image-base
no additional variables

The traceback error msg that I get when I press the build button:

2016-10-20 08:45:34,486 ERROR runbuilds: Error launching build 'ascii' codec can't decode byte 0xc2 in position 10: ordinal not in range(128)
Traceback (most recent call last):
  File "/home/anatoli/compulab-bsp/sources/poky/bitbake/lib/toaster/bldcontrol/management/commands/runbuilds.py", line 67, in schedule
    "%d:%d" % (br.pk, bec.be.pk))
  File "/home/anatoli/compulab-bsp/sources/poky/bitbake/lib/toaster/bldcontrol/localhostbecontroller.py", line 255, in triggerBuild
    layers = self.setLayers(bitbake, layers, targets)
  File "/home/anatoli/compulab-bsp/sources/poky/bitbake/lib/toaster/bldcontrol/localhostbecontroller.py", line 155, in setLayers
    self._shellcmd('git clone "%s" "%s"' % (giturl, localdirname))
  File "/home/anatoli/compulab-bsp/sources/poky/bitbake/lib/toaster/bldcontrol/localhostbecontroller.py", line 68, in _shellcmd
    err = "command: %s \n%s" % (command, err)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc2 in position 10: ordinal not in range(128)
2016-10-20 08:45:35,533 DEBUG Saving error 'ascii' codec can't decode byte 0xc2 in position 10: ordinal not in range(128) (cm-fx6-toaster (Release krogoth (krogoth), BBV krogoth (Branch: krogoth)) failed)
Comment 5 brian avery 2016-11-09 22:16:24 UTC
Hmm, we are having trouble reproducing this.
Thanks for the additional information!

A couple of things,
0) in krogoth you get a server bound to 0.0.0.0, which will allow you to access the server from a different machine. in morty, it defaults to 127.0.0.1:8000, so it wil only be accessible locally unless you add webport=0.0.0.0:8000 to the toaster start command.
1) Could you try the new release? it is called morty.  We did a switchover to Python 3.4 in the Morty release and cleaned up some (but i fear not all) of the unicode issues.
2) it looks like the command that is failing is     self._shellcmd('git clone "%s" "%s"' % (giturl, localdirname))
and it looks like a non ascii character is in one of these.  Any idea where it's creeping in?

3) I couldn't replicate your error on krogoth. Where doed meta-compulab come from?  did you pull all the other layers in usig toaster and layers.openembedded.org?
Comment 6 brian avery 2016-11-14 21:02:01 UTC
waiting for more info so I can reproduce
Comment 7 brian avery 2016-11-28 19:12:48 UTC
Not enough information to reproduce the specific occurrence of the bug.  The general work of insuring unicode support throughout toaster is still a good idea.
Comment 8 brian avery 2017-06-15 15:10:47 UTC
failed to reproduce :(