Mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Muhammad Haris <muhammad.haris.makh...@gmail.com>
Subject Re: Tez Job Counters
Date Tue, 28 Jun 2016 20:20:35 GMT
Hi,
Could anybody please guide me how to get all task level counters? Thanks a
lot



Regards

On Mon, Jun 27, 2016 at 5:43 PM, Muhammad Haris <
muhammad.haris.makhtar@gmail.com> wrote:

> Hi Rajesh,
> Thank you for quick help
>
>
> Regards
>
> On Mon, Jun 27, 2016 at 4:56 PM, Rajesh Balamohan <rbalamohan@apache.org>
> wrote:
>
>> You can refer to Tez-UI which makes use of these APIs extensively.
>> https://tez.apache.org/tez-ui.html provides
>> details on setting this up.
>>
>> AppID:
>> http://atsmachine:8088/ws/v1/cluster/apps/application_1466689310983_0024
>>
>> Tez DAG:
>> http://atsmachine:8188/ws/v1/timeline/TEZ_DAG_ID?limit=10
>>
>> http://atsmachine:8188/ws/v1/timeline/TEZ_DAG_ID/dag_1466689310983_0021_409
>>
>>
>> This should have the counters in the otherInfo --> counters -->
>> counterGroups
>>
>> Tez vertex:
>>
>> http://atsmachine:8188/ws/v1/timeline/TEZ_VERTEX_ID?limit=10&primaryFilter=TEZ_DAG_ID:dag_1466689310983_0023_408
>>
>> Tez task:
>>
>> http://atsmachine:8188/ws/v1/timeline/TEZ_TASK_ID?limit=1000&primaryFilter=TEZ_DAG_ID:dag_1466689310983_0023_408
>>
>> Tez task attempt:
>>
>> http://atsmachine:8188/ws/v1/timeline/TEZ_TASK_ATTEMPT_ID?limit=1000&primaryFilter=TEZ_DAG_ID:dag_1466689310983_0023_408
>>
>> ~Rajesh.B
>>
>> On Mon, Jun 27, 2016 at 3:50 AM, Muhammad Haris <
>> muhammad.haris.makhtar@gmail.com> wrote:
>>
>>> Hi,
>>> I have implemented an application that runs Hive jobs and pulls its job
>>> counters using Job History server REST API, Job History server REST API
>>> that i am using is:
>>> http://<history server http
>>> address:port>/ws/v1/history/mapreduce/jobs/{jobid}/counters
>>>
>>> I have been looking for similar REST API to get job counters for Hive on
>>> Tez jobs, i understand Tez jobs history is pushed into YARN timeline server
>>> but i failed to find any API in YARN timeline server through which i can
>>> pull the job counters.
>>>
>>> Any help/suggestions will be highly appreciated
>>> Regards
>>> Haris Akhtar
>>>
>>>
>>
>

Mime
View raw message