Raadiolained, moduleerimine, induktsooon

Kogun siia enda jaoks subjektis toodu teemaga seotud linke ja infot, et neist ükspäeb enda jaoks terviklik pilt kokku panna.

Sageduse suurenedes suureneb aktiivtakistus pindefekti mõjul – juhtmes indutseeritud pöörisvoolude mõjul kulgeb vool rohkem pinnakihtides. Juhtme südamik jääb põhiliselt kasutamata, seetõttu juhtme ristlõikepind näivalt väheneb ja takistus suureneb. (Allikas: http://www.ene.ttu.ee/leonardo/elektro_alused/6Vahelduvvool.pdf)

 

 

 

Allikas: https://commons.wikimedia.org/wiki/Main_Page

 

 

https://www.electronics-tutorials.ws/oscillator/oscillators.html

http://www.learnabout-electronics.org/Oscillators/osc10.php

Phoenix org.apache.phoenix.mapreduce.index.IndexTool java.lang.ArrayIndexOutOfBoundsException

One good example how community works!

I had a strange error:

Hi

Needed to recreate indexes over main table contains more than 2.3 x 10^10 records.
I used ASYNC and org.apache.phoenix.mapreduce.index.IndexTool
One index succeed but another gives stack:
2018-03-20 13:23:16,723 FATAL [IPC Server handler 0 on 43926] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Task: attempt_1521544097253_0004_m_000008_0 – exited : java.lang.ArrayIndexOutOfBoundsException at org.apache.hadoop.mapred.MapTask$MapOutputBuffer$Buffer.write(MapTask.java:1453) at org.apache.hadoop.mapred.MapTask$MapOutputBuffer$Buffer.write(MapTask.java:1349) at java.io.DataOutputStream.writeInt(DataOutputStream.java:197) at org.apache.hadoop.hbase.io.ImmutableBytesWritable.write(ImmutableBytesWritable.java:159) at org.apache.hadoop.io.serializer.WritableSerialization$WritableSerializer.serialize(WritableSerialization.java:98) at org.apache.hadoop.io.serializer.WritableSerialization$WritableSerializer.serialize(WritableSerialization.java:82) at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.collect(MapTask.java:1149) at org.apache.hadoop.mapred.MapTask$NewOutputCollector.write(MapTask.java:715) at org.apache.hadoop.mapreduce.task.TaskInputOutputContextImpl.write(TaskInputOutputContextImpl.java:89) at org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.write(WrappedMapper.java:112) at org.apache.phoenix.mapreduce.index.PhoenixIndexImportMapper.map(PhoenixIndexImportMapper.java:114) at org.apache.phoenix.mapreduce.index.PhoenixIndexImportMapper.map(PhoenixIndexImportMapper.java:48) at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:146) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:787) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341) at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:170) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1866) at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:164)
 
 
Is here any best practice how to deal with situations like this?
 
Br, Margus
And there was answer out there!
Hard to say at a glance, but this issue is happening down in the MapReduce framework, not in Phoenix itself.

It looks similar to problems I’ve seen many years ago around mapreduce.task.io.sort.mb. You can try reducing that value. It also may be related to a bug in your Hadoop version.

And it resolved my problem!
Great hint! Looks like it helped!

What a great power of community!

Br, Margus

Posted in IT

Game Theory

Aeg-ajalt olen seda fraasi kuulnud ja siis googeldanud ja üldmõttest aru saanud. Et edaspidi lihtsam oleks, siis kirjutan omale siia maha kokkuvõtte mänguteeooria ühest levinumast näitest – Prisoners’ Dilemma.

Olukord on siis selline, et kaks tegelast jäävad autovargusega vahele, mille eest tahetakse neile panna karistuseks kaks aastat. Mõlemad paigutatakse eraldi kambritesse ülekuulamiseks. Samas selgub, et mõlemat tegelast kahtlustatakse ka pangaröövis. Tegelased omavahel suhelda ei saa.

Mõlemale pätile antakse valikud:

  1. Kui mõlemad tunnistavad pangaröövi ülesse, siis saavad kolm aastat.
  2. Kui üks tunnistab pangaröövi ülesse ja teine ei tunnista, siis see, kes tunnistab pangaröövi saab ühe aasta ja see, kes ei tunnistanud saab kümme aastat.

Antud valikutest saab moodustada maatriksi:

Vang 1
TunnistabEi tunnista
Vang 2TunnistabMõlemad saavad kolm aastatVang 2 saab ühe aasta. Vang 2 saab kümme aastat.
Ei tunnistaVang 1 saab kümme aastat. Vang 1 saab ühe aastaVang 1 saab kaks aastat. Vang 2 saab kaks aastat.

Kui nüüd mõlemad vangid läheksid oma karistuse minimaliseerimise peale (kaks aastat), siis on oht, et üks või teine tunnistab ennast siiski pangaröövis süüdi ja üks või teine saab kümme aastat. Analüüsides ülal toodud lihtsat tabelit, siis selgub, oluline on mõlema vangi vaatevinklist kümme aastal laualt minema saada. Ja valikutesse jääb üks aasta, juhul kui üks või teine ei tunnista ennast süüdi või siis kolm aastat kui mõlemad tunnistavad ennast süüdi.

Scala higher-order functions

For me personally, very difficult to thing recursively. So I’ll put some examples for me down here.

First – higher-order function is a function takes function as an argument or returns result as a function.

Lets create a high-order function in scala:

scala> def highF(f: Int => Int, a: Int) = f(a)
highF: (f: Int => Int, a: Int)Int

As we can see it does not do anything usable. Just takes f(Int) and returns Int.

Lets create two first class functions:

scala> def first1(a: Int) : Int = { a.+(a) }
first1: (a: Int)Int

scala> def first2(a: Int) : Int = { a.-(a) }
first2: (a: Int)Int

This functions are already easier to explain. First one takes Int and just does Int + Int. In example a => a + a.

Second one just subtracts a => a – a.

Now comes interesting part – how we use higher-order function.

scala> highF(first1, 2)
res23: Int = 4

scala> highF(first2, 2)
res24: Int = 0

And there is more. We can use anonymous function as an argument:

scala> highF(a => a.+(10), 2)
res25: Int = 12

Posted in IT

What does reduce

 

List( 1, 2, 3, 4 ).reduce( (x,y) => x + y )   == (((1+2)+3)+4)
Step 1 : op( 1, 2 ) will be the first evaluation. 
  Start with 1, 2, that is 
    x is 1  and  y is 2
Step 2:  op( op( 1, 2 ), 3 ) - take the next element 3
  Take the next element 3: 
    x is op(1,2) = 3   and y = 3
Step 3:  op( op( op( 1, 2 ), 3 ), 4) 
  Take the next element 4: 
    x is op(op(1,2), 3 ) = op( 3,3 ) = 6    and y is 4

Restore ESP8266EX original firmware using esptool

https://github.com/mlwmlw/esp8266-workshop/tree/master/firmware

margusja@IRack:/System/Library/Extensions$ esptool.py –port /dev/tty.wchusbserial1410 write_flash 0x00000 /Users/margusja/Downloads/v1.3.0.2\ AT\ Firmware.bin
esptool.py v2.2
Connecting….
Detecting chip type… ESP8266
Chip is ESP8266EX
Uploading stub…
Running stub…
Stub running…
Configuring flash size…
Auto-detected Flash size: 1MB
Compressed 1044480 bytes to 242823…
Wrote 1044480 bytes (242823 compressed) at 0x00000000 in 21.6 seconds (effective 387.1 kbit/s)…
Hash of data verified.

Leaving…
Hard resetting…

 

AT+GMR
AT version:0.40.0.0(Aug 8 2015 14:45:58)
SDK version:1.3.0
Ai-Thinker Technology Co.,Ltd.
Build:1.3.0.2 Sep 11 2015 11:48:04
OK

FlameGraph

Thanks to Erkki I found project FlameGraph.

Just to remember to myself one session:

su – hdfs

28 git clone https://github.com/jvm-profiling-tools/async-profiler
29 git clone https://github.com/BrendanGregg/FlameGraph
30 cd async-profiler
33 JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk/ make
34 cd ../
35 mkdir async-profiler-output
36 cd async-profiler-output
37 jps (get hdfs datanode process)
38 ../async-profiler/profiler.sh -t -d 10 -o collapsed -f /tmp/collapsed.txt 231591
41 ../FlameGraph/flamegraph.pl –colors=java /tmp/collapsed.txt > flamegraph_yarn-bigdata40_hdfs_namenode.svg

 

Got nice flame

Posted in IT

OpenCP cv:Mat and cv:transpose

Matrix transpose does some cool stuff with matrix elements. In example using OpenCV:

    Mat A = (Mat_<float>(2, 5) << 1, 2, 3, 4, 5, 7, 8, 9, 10, 11);

    cout << “A = “ << endl << ” “ << A << endl << endl;

    Mat A1;

    transpose(A, A1);

    cout << “A1 = “ << endl << ” “ << A1 << endl << endl;

A =

[1, 2, 3, 4, 5;

7, 8, 9, 10, 11]

 

A1 =

[1, 7;

2, 8;

3, 9;

4, 10;

5, 11]

 

Nice yeah.

As OpenCV holds pictures as matrixes then lets find out what is turns out to transpose image.

 

 

Picture has a little more matrix elements than in a previous example –

Image heigth: 720 cols: 1080 – It means 720 x 1080 elements.

    // Load an color image in grayscale

    Mat img = imread(“/Users/margusja/Pictures/faces/margusja2.jpg”,0);

    cv::Size s = img.size();

    int rows = s.height;

    int cols = s.width;

    Mat fimage;

    transpose(img, fimage);

    cout << “Image heigth: “ << rows << ” cols: “ << cols << endl;

    namedWindow( “Display window”, WINDOW_AUTOSIZE );

    imshow(“Margusja 1”,img);

    namedWindow( “Display window”, WINDOW_AUTOSIZE );

    imshow(“Margusja transposed”,fimage);

And result:

Versus

 

Now we have a clue how they rotate our pictures 🙂

Simple Tensorflow arithmetic

Lets imagine we have to do a simple arithmetic: (10+20) * (30-40)

In some unclear reason we decidet to use Tensorflow.

Default language is there python.

(tensorflow) margusja@IRack:~/tensorflow/tensorflow_scripts$ python
Python 3.6.0 |Continuum Analytics, Inc.| (default, Dec 23 2016, 13:19:00)
[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.57)] on darwin
Type “help”, “copyright”, “credits” or “license” for more information.
>>> import tensorflow as tf
/Users/margusja/tensorflow/lib/python3.6/importlib/_bootstrap.py:205: RuntimeWarning: compiletime version 3.5 of module ‘tensorflow.python.framework.fast_tensor_util’ does not match runtime version 3.6
return f(*args, **kwds)
>>>
>>> a = tf.constant(10)
>>> b = tf.constant(20)
>>> c = tf.constant(30)
>>> d = tf.constant(40)
>>> e = tf.add(a,b)
>>> f = tf.subtract(c,d)
>>> h = tf.multiply(e,f)
>>> sess = tf.Session()
2017-11-10 19:02:43.777454: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
>>> print(sess.run(h))
-300

We can explore graph in tensorbar:

 

my vim tips

  1. :sp filename for a horizontal split
  2. :vsp filename or :vs filename for a vertical split

move between splits: CNTR+w and arrow

Posted in IT